Forecast-Based Model Selection in the Presence of Structural Breaks

FRB of Kansas City Working Paper No. 02-05

52 Pages Posted: 23 Nov 2002

See all articles by Todd E. Clark

Todd E. Clark

Federal Reserve Bank of Cleveland

Michael W. McCracken

Federal Reserve Banks - Federal Reserve Bank of St. Louis

Date Written: August 2002

Abstract

This paper presents analytical, Monte Carlo, and empirical evidence on the effects of structural breaks on tests for equal forecast accuracy and forecast encompassing. The forecasts are generated from two parametric, linear models that are nested under the null. The alternative hypotheses allow a causal relationship that is subject to breaks during the sample. With this framework, we show that in-sample explanatory power is readily found because the usual F-test will indicate causality if it existed for any portion of the sample. Out-of-sample predictive power can be harder to find because the results of out-of-sample tests are highly dependent on the timing of the predictive ability. Moreover, out-of-sample predictive power is harder to find with some tests than with others: The power of F-type tests of equal forecast accuracy and encompassing often dominates that of the more commonly-used t-type alternatives. Overall, out-of-sample tests are effective at revealing whether one variable has predictive power for another at the end of the sample. Based on these results and additional evidence from two empirical applications, we conclude that structural breaks can explain why researchers often find evidence of in-sample, but not out-of-sample, predictive content.

Keywords: Power, Structural Breaks, Forecast Evaluation, Model Selection

JEL Classification: C53, C12, C52

Suggested Citation

Clark, Todd E. and McCracken, Michael W., Forecast-Based Model Selection in the Presence of Structural Breaks (August 2002). FRB of Kansas City Working Paper No. 02-05, Available at SSRN: https://ssrn.com/abstract=349061 or http://dx.doi.org/10.2139/ssrn.349061

Todd E. Clark (Contact Author)

Federal Reserve Bank of Cleveland ( email )

P.O. Box 6387
Cleveland, OH 44101
United States
216-579-2015 (Phone)

Michael W. McCracken

Federal Reserve Banks - Federal Reserve Bank of St. Louis ( email )

411 Locust St
Saint Louis, MO 63011
United States