Nested Forecast Model Comparisons: A New Approach to Testing Equal Accuracy

FRB of St. Louis Working Paper No. 2009-050B

61 Pages Posted: 24 Oct 2012

See all articles by Todd E. Clark

Todd E. Clark

Federal Reserve Bank of Cleveland

Michael W. McCracken

Federal Reserve Banks - Federal Reserve Bank of St. Louis

Multiple version iconThere are 2 versions of this paper

Date Written: August 2012

Abstract

This paper develops bootstrap methods for testing whether, in a finite sample, competing out-of-sample forecasts from nested models are equally accurate. Most prior work on forecast tests for nested models has focused on a null hypothesis of equal accuracy in population — basically, whether coefficients on the extra variables in the larger, nesting model are zero. We instead use an asymptotic approximation that treats the coefficients as non-zero but small, such that, in a finite sample, forecasts from the small model are expected to be as accurate as forecasts from the large model. Under that approximation, we derive the limiting distributions of tests of equal mean square error, and develop bootstrap methods for estimating critical values. Monte Carlo experiments show that our proposed procedures have good size and power properties for the null of equal finite-sample forecast accuracy. We illustrate the use of the procedures with applications to forecasting stock returns and inflation.

Keywords: mean square error, prediction, reality check

JEL Classification: C53, C12, C52

Suggested Citation

Clark, Todd E. and McCracken, Michael W., Nested Forecast Model Comparisons: A New Approach to Testing Equal Accuracy (August 2012). FRB of St. Louis Working Paper No. 2009-050B, Available at SSRN: https://ssrn.com/abstract=2166366 or http://dx.doi.org/10.2139/ssrn.2166366

Todd E. Clark

Federal Reserve Bank of Cleveland ( email )

P.O. Box 6387
Cleveland, OH 44101
United States
216-579-2015 (Phone)

Michael W. McCracken (Contact Author)

Federal Reserve Banks - Federal Reserve Bank of St. Louis ( email )

411 Locust St
Saint Louis, MO 63011
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
125
Abstract Views
1,073
Rank
264,034
PlumX Metrics