If Nonlinear Models Cannot Forecast, What Use Are They?
Studies in Nonlinear Dynamics & Econometrics
This paper begins with a brief review of the recent experience using nonlinear models and ideas of chaos to model economic data and to provide forecasts that are better than linear models. The record of improvement is at best meager. The remainder of the paper examines some of the reasons for this lack of improvement. The concepts of "openness" and "isolation" are introduced, and a case is made that open and nonisolated systems cannot be forecasted; the extent to which economic systems are
... ic systems are closed and isolated provides the true pragmatic limits to forecastability. The reasons why local "overfitting," especially with nonparametric models, leads to worse forecasts are discussed. Models and "representations" of data are distinguished and the reliance on minimum mean-square forecast error to choose between models and representations is evaluated. , 1(2): 65-86 amorphous collection of models defined by exclusion from the well-defined class of linear models. Nonlinear models are usually modeled by nonlinear differential, or difference, equations. Models of deterministic chaos are a subcategory of nonlinear models that are characterized by the conjunction of local instability, thereby giving rise to sensitivity to initial conditions, with global stability that effectively restricts the long-term domain of the dynamic orbits to a bounded compact set. Chaotic paths, notwithstanding their seemingly random time paths, are in fact representations of long-term steady states of the corresponding dynamical system. A second major strand to use of the term "nonlinear" is in the context of stochastic models. In this situation, the adjective "nonlinear" may refer to properties of models that link the conditional moments of the distributions involved to the time paths of exogenous variables, or to the nature of the time-varying path of the distributions themselves. With respect to the former concept, researchers frequently restrict attention to just the first two moments, although this is more a matter of convenience than of theoretical necessity or empirical fact. In the subsequent development, I will use the term "nonlinear" in all of its various senses, but will endeavor to be clear which is which at any given point in the text. Forecasting and Nonlinear Models: The Record The empirical literature on chaotic and nonlinear dynamics began in about 1986. Since that time, there has been an enormous amount of activity. The following comments cannot hope to provide a thorough review of the empirical literature. I will restrict myself to a brief schematic review indicating only the main results, the "bottom line," as it were. In the discussion below I will refer to a few reviews of the literature which do an excellent job of bringing the new reader up to date; see in particular Lorenz (1993 ), Bollerslev et al. (1990 ), Brock and Potter (1993 , and Le Baron (1994). The initial flurry of activity investigating the role of "chaos" in economics that began in 1986 centered on calculating Lyapunov exponents and dimension numbers. Lyapunov exponents determine the degree of local instability of the time paths of a dynamic system, and dimension indicates the degree of complexity of the system as determined by the minimum number of variables that is needed to model the system. An objective of the early analysis was to discover the presence of a deterministic chaotic structure that would be analogous to the discoveries in the experimental sciences. Unfortunately, the challenge was too great. The sparsity of data was a problem, albeit not the most important one. There were also difficulties caused by aggregation and the fact that the sampling rate is usually far too coarse for detecting fine dynamical structure; see for example, Ramsey and Yuan (1989) and Ramsey et al. (1990) . Some of the other reasons for the "nondetection" of chaos will be enunciated below when I discuss the concepts of openness and nonisolation of systems. A more germane reason for the failure was that in the experimental sciences the empirical discovery of chaos was always achieved by examining systems that were on the boundary of the transition from periodicity to chaos. If we were to examine systems that are deeply into the chaotic regime, we would discover that the tools available for the detection of chaos are inadequate even in the experimental sciences, Casdagli (1989) . What is needed to discover chaos in economics is a series of experiments that will enable one to evaluate behavior as the system is "forced" to undergo a phase transition from periodicity to chaos. Actual economic systems are not likely to be observed on the borderline between periodicity and chaos. Thus, the most likely potential for the discovery of chaos in economics is through experimental economics. The outcome of the effort to discover chaos in the context of economic and financial data is best summarized in the words of Granger and Teräsvirta (1992) : "Deterministic (chaotic) models are of little relevance in economics and so we will consider only stochastic models." The theme was echoed and amplified by Jaditz and Sayers (1992), who reviewed a wide variety of research to conclude that there was no evidence for chaos, but that was not to deny the indication of nonlinear dynamics of some sort (Brock and Potter 1993; Le Baron 1994; and Ashley and Patterson 1989) . The more limited objective of finding a data-dependent method for determining the "dimension" of an economic system has been an equal failure so far. The importance of this task is clear. If one can obtain an estimate of the number of degrees of freedom within an economic system, then the task of the econometrician is greatly simplified; and if, for reasons to be discussed later, the number of degrees of freedom tends to vary by small amounts over time, the econometric benefits from dimension estimation would be even greater. Unfortunately, the determination of the number of degrees of freedom of an economic system is no easy task, and it cannot be accomplished by a straightforward calculation of dimension constants, no matter how defined (Casdagli 1992; Hunter 1992) .