A Survey of Recent Advances in Forecast Accuracy Comparison Testing, With an Extension to Stochastic Dominance
Recent Advances and Future Directions in Causality, Prediction, and Specification Analysis: Essays in Honor of Halbert L. White Jr., Springer, 2012
26 Pages Posted: 15 Jul 2013
Date Written: February 2012
Abstract
In recent years, an impressive body or research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (DM: 1995) that sets the groundwork for much of the subsequent work in the area, West (1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution (Corradi and Swanson 2006a,b) and of an unknown conditional distribution (Corradi and Swanson 2007b). Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance; and we choose the model whose CDF is stochastically dominated, over some given range of interest.
Keywords: block bootstrap, recursive estimation scheme, reality check, parameter estimation error
JEL Classification: C22, C51
Suggested Citation: Suggested Citation
Do you have negative results from your research you’d like to share?
Recommended Papers
-
Stepwise Multiple Testing as Formalized Data Snooping
By Michael Wolf and Joseph P. Romano
-
Improved Nonparametric Confidence Intervals in Time Series Regressions
By Joseph P. Romano and Michael Wolf
-
By James J. Heckman, Seong Moon, ...
-
Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing
By Joseph P. Romano and Michael Wolf
-
Balanced Control of Generalized Error Rates
By Joseph P. Romano and Michael Wolf
-
Control of the False Discovery Rate Under Dependence Using the Bootstrap and Subsampling
By Joseph P. Romano, Azeem Shaikh, ...
-
Hypothesis Testing in Econometrics
By Joseph P. Romano, Azeem Shaikh, ...
-
Inference with Imperfect Randomization: The Case of the Perry Preschool Program
By James J. Heckman, Rodrigo R. Pinto, ...
-
Primate Evidence on the Late Health Effects of Early Life Adversity
By Gabriella Conti, Christopher Hansman, ...
-
By Ashok Kaul and Michael Wolf