Development and validation of a rule-based time series complexity scoring technique to support design of adaptive forecasting DSS

Monica Adya, Edward J. Lusk
2016 Decision Support Systems  
Evidence from forecasting research gives reason to believe that understanding time series complexity can enable design of adaptive forecasting decision support systems (FDSSs) to positively support forecasting behaviors and accuracy of outcomes. Yet, such FDSS design capabilities have not been formally explored because there exists no systematic approach to identifying series complexity. This study describes the NOT THE PUBLISHED VERSION; this is the author's final, peer-reviewed manuscript.
more » ... published version may be accessed by following the link in the citation at the bottom of the page. 2 development and validation of a rule-based complexity scoring technique (CST) that generates a complexity score for time series using 12 rules that rely on 14 features of series. The rule-based schema was developed on 74 series and validated on 52 holdback series using well-accepted forecasting methods as benchmarks. A supporting experimental validation was conducted with 14 participants who generated 336 structured judgmental forecasts for sets of series classified as simple or complex by the CST. Benchmark comparisons validated the CST by confirming, as hypothesized, that forecasting accuracy was lower for series scored by the technique as complex when compared to the accuracy of those scored as simple. The study concludes with a comprehensive framework for design of FDSS that can integrate the CST to adaptively support forecasters under varied conditions of series complexity. The framework is founded on the concepts of restrictiveness and guidance and offers specific recommendations on how these elements can be built in FDSS to support complexity. context and characteristics, cognitive needs of forecasters, and patterns of information use 5,6 thereby debiasing the decision process. 7 The design of such adaptive systems for forecasting, however, has remained unexplored as there exists no formal way of characterizing the complexity of forecasting tasks. Fully disclosing development of the CST provides opportunities for further validation and refinement at many levels. Implications for judgmental forecasting and AFDSS design are numerous and, as such, the CST seeds a new stream of forecasting research on series complexity and supporting processes.
doi:10.1016/j.dss.2015.12.009 fatcat:6wq7hzerajd4vhuxoucuhlhade