ONLINE MULTI-TASK LEARNING FRAMEWORKFOR ENSEMBLE FORECASTING
ABSTRACT
Ensemble forecasting is a widely-used numerical prediction method for modeling the evolution of nonlinear dynamicsystems. To predict the future state of such systems, a set of ensemble member forecasts is generated from multiple runs of computermodels, where each run is obtained by perturbing the starting condition or using a different model representation of the system. Theensemble mean or median is typically chosen as a point estimate for the ensemble member forecasts. These approaches are limited inthat they assume each ensemble member is equally skillful and may not preserve the temporal autocorrelation of the predicted timeseries. To overcome these limitations, we present an online multi-task learning framework called ORION to estimate the optimalweights for combining the ensemble member forecasts. Unlike other existing formulations, the proposed framework is novel in that itslearning algorithm must backtrack and revise its previous forecasts before making future predictions if the earlier forecasts wereincorrect when verified against new observation data. We termed this strategy as online learning with restart. Our proposed frameworkemploys a graph Laplacian regularizer to ensure consistency of the predicted time series. It can also accommodate different types ofloss functions, including _-insensitive and quantile loss functions, the latter of which is particularly useful for extreme value prediction. Atheoretical proof demonstrating the convergence of our algorithm is also given. Experimental results on seasonal soil moistureforecasts from 12 major river basins in North America demonstrate the superiority of ORION compared to other baseline algorithms.
EXISTING SYSTEM:
This section reviews some of the previous works on topicsclosely related to this paper. An ensemble forecasting taskrequires predicting the future values of a time series over afinite time window, which is quite similar to the multi-stepahead time series prediction problem. Nevertheless,there is a fundamental difference between the two predictionproblems. Multi-step-ahead time series predictionmethods consider only the historical values of a time seriesto infer its future values. Thus, it is susceptible to the erroraccumulation problem. In contrast, ensemble forecastingmethods employ multivariate time series generated fromcomputer models to predict the future values of a time series.These models generate their outputs by considering thephysical processes that govern the evolution of the dynamicsystem.
PROPOSED SYSTEM:
The main contributions of this paper are as follows:_ We introduce the problem of online regularizedmulti-task regression with partially observed dataand demonstrate its relevance to the ensemble forecastingtask._ We present a novel framework called ORION, whichuses an online learning with restart strategy to solvethe problem. It also uses a graph Laplacian to capturerelationships a the learning tasks alongwith a passive aggressive update scheme to optimizethe _-insensitive loss function._ We extended the framework to incorporate a quantileloss function for predicting extreme events. Tothe best of our knowledge, ORION is the first multitaskregression framework that has been tailored forextreme value prediction._ We performed extensive experiments using a realworldsoil moisture data set and showed that ORIONoutperforms several baseline algorithms, includingthe ensemble median, for the majority of the riverbasins in our data set.
CONCLUSION
This paper presents an online regularized multi-task regressionframework for ensemble forecasting tasks. Our frameworkis unique in that it uses an online learning with restartstrategy to update its models. The proposed framework isalso flexible in that it can accommodate both _-insensitiveand quantile loss functions. Experimental results confirmthe superiority of the proposed framework compared toseveral baseline methods.
REFERENCES
[1] C. Tebaldi and R. Knutti, “The use of the multi-model ensemble inprobabilistic climate projections,” Philosoph. Trans. Roy. Soc. LondonA: Math. Phys. Eng. Sci., vol. 365, no. 1857, pp. 2053–2075, 2007.
[2] P. Cantelaube and J.-M. Terres, “Seasonal weather forecasts forcrop yield modeling in Europe,” Tellus A, vol. 57, no. 3, pp. 476–487, 2005.
[3] M. B. Araujo and M. New, “Ensemble forecasting of species distributions,”Trends Ecology Evol., vol. 22, no. 1, pp. 42–47, 2007.
[4] L. Luo and E. F. Wood, “Use of Bayesian merging techniques in amultimodel seasonal hydrologic ensemble prediction system forthe Eastern United States,” J. Hydrometeorology, vol. 9, pp. 866–884, 2008.
[5] M. Leutbecher and T. Palmer, “Ensemble forecasting,” J. Comput.Phys., vol. 227, pp. 3515–3539, 2008.
[6] X.-T. Yuan and S. Yan, “Visual classification with multi-task jointsparse representation,” in Proc. IEEE Comput. Soc. Conf. Comput.Vis. Pattern Recognit., 2010, pp. 3493–3500.
[7] X. Wang, C. Zhang, and Z. Zhang, “Boosted multi-task learningfor face verification with applications to Web image and videosearch,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009,pp. 142–149.
[8] J. Zhou, L. Yuan, J. Liu, and J. Ye, “A multi-task learning formulationfor predicting disease progression,” in Proc. 17th ACMSIGKDD Int. Conf. Knowl. Discovery Data Mining, 2011, pp. 814–822.
[9] S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, “Multi-tasklearning for HIV therapy screening,” in Proc. 25th Int. Conf. Mach.Learn., 2008, pp. 56–63.
[10] X. Ning and G. Karypis, “Multi-task learning for recommendersystem,” in Proc. 2nd Asian Conf. Mach. Learn., 2010, pp. 269–284.
[11] J. Wang, S. C. Hoi, P. Zhao, and Z.-Y. Liu, “Online multi-task collaborativefiltering for on-the-fly recommender systems,” in Proc.7th ACM Conf. Recommender Syst., 2013, pp. 237–244.