NN3 NN5
The NN3 Competition (NN3 2007, Crone et al. 2011) was devised with two datasets composed of monthly time series drawn from homogeneous population of real empirical business time series.
Dataset A contains 111 different monthly time series. Each of this time series possess from 50 to 126 observations. Dataset B is actually a sample of dataset A containing its last 11 monthly time series. The last three time series of dataset A, also presented in dataset B, are displayed in Fig. 1, in which the difference between the traits of the time series is easily noticeable.
Each competitor in NN3 was asked to choose one of the two datasets and predict the next 18 corresponding observations.
Fig. 1 Examples of 3 time series retrieved from the complete dataset (dataset A) of the NN3 Competition. The graphics (a), (b) and (c) correspond to the NN3.109, NN3.110 and NN3.111 time series, respectively.
The NN5 Competition (NN5 2008, Crone 2008) was devised to extend the earlier NN3 prediction competition. Like NN3, NN5 offers two datasets. Dataset A is the complete dataset with 111 daily time series, and dataset B is composed of a sample of dataset A containing its last 11 daily time series.
The time series are originated from the observation of daily withdrawals at 111 randomly selected different cash machines at different locations within England.
However, differently from The NN3 Competition, each of these time series is considerably larger, possessing 735 observations, and may present missing data. The time series also shows different patterns of single or multiple overlying seasonal properties. Examples of four time series of the dataset A are presented in Fig. 2, in which one may observe some of the cited characteristics. We note that the time series represented in Fig. 2.c and Fig. 2.d are also present in dataset B.
Each competitor was asked to choose one of these two datasets and predict the next 56 observations for each time series.
Fig. 2 Examples of 4 time series retrieved from the complete dataset (Dataset A) of the NN5 Competition. The graphics (a), (b), (c) and (d) correspond to the NN5.002, NN5.003, NN5.110 and NN5.111 time series, respectively.
The performance evaluations done by NN3 and NN5 Competitions were based on the mean SMAPE error of prediction found by the competitors across all time series.
The complete dataset A of the NN3 and NN5 Competitions are present in TSPred R-Package as NN3.A and NN5.A, respectively. The values which were to be predicted of these two time series datasets are also present as NN3.A.cont and NN5.A.cont, respectively.
#Load the datasets NN3.A and NN3.A.cont
> data(NN3.A,NN3.A.cont)
#Automatically fits an ARIMA model to each times series in NN3.A and predicts the values in NN3.A.cont
> pred <- marimapred(NN3.A,NN3.A.cont,plot=FALSE)
#Calculates the mean SMAPE error of prediction between pred and NN3.A.cont
> sMAPEVector <- mapply(sMAPE, NN3.A.cont, data.frame(pred), SIMPLIFY = TRUE, USE.NAMES = TRUE)
#Binds the SMAPE errors with respect to Dataset A and Dataset B in a vector
> cbind(MeanSMAPE111=mean(sMAPEVector)*100, MeanSMAPE11=mean(tail(sMAPEVector,11))*100)
#Load the datasets NN5.A and NN5.A.cont
> data(NN5.A,NN5.A.cont)
#Automatically fits an ARIMA model to each times series in NN5.A and predicts the values in NN5.A.cont
> pred <- marimapred(NN5.A,NN5.A.cont,plot=FALSE)
#Calculates the mean SMAPE error of prediction between pred and NN5.A.cont
> sMAPEVector <- mapply(sMAPE, NN5.A.cont, data.frame(pred), SIMPLIFY = TRUE, USE.NAMES = TRUE)
#Binds the SMAPE errors with respect to Dataset A and Dataset B in a vector
> cbind(MeanSMAPE111=mean(sMAPEVector)*100, MeanSMAPE11=mean(tail(sMAPEVector,11))*100)
> ARIMA.NN3.NN5 <- function(TimeSeries, TimeSeriesCont, plot=FALSE){
if(is.null(TimeSeries)) stop("TimeSeries is required and must have positive length")
if(is.null(TimeSeriesCont)) stop("TimeSeriesCont is required and must have positive length")
Predictions <- marimapred(TimeSeries, TimeSeriesCont, plot=plot)
sMAPEVector <- mapply(sMAPE, TimeSeriesCont, data.frame(Predictions), SIMPLIFY = TRUE, USE.NAMES = TRUE)
return (cbind(MeanSMAPE111=mean(sMAPEVector)*100, MeanSMAPE11=mean(tail(sMAPEVector,11))*100))
}
> ARIMA.NN3.NN5(NN3.A,NN3.A.cont)
> ARIMA.NN3.NN5(NN5.A,NN5.A.cont)
S.F. Crone, M. Hibon, and K. Nikolopoulos, 2011, Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction, International Journal of Forecasting, v. 27, n. 3 (Jul.), p. 635–660.
NN3 2007, The NN3 Competition: Forecasting competition for artificial neural networks and computational intelligence. URL: http://www.neural-forecasting-competition.com/NN3/index.htm.
S.F. Crone, 2008, Results of the NN5 time series forecasting competition. Hong Kong, Presentation at the IEEE world congress on computational intelligence. WCCI’2008.
NN5 2008, The NN5 Competition: Forecasting competition for artificial neural networks and computational intelligence. URL: http://www.neural-forecasting-competition.com/NN5/index.htm.