Forecast Evaluation with Shared Data Sets

24 Pages Posted: 2 Dec 2001

See all articles by Ryan Sullivan

Ryan Sullivan

Bates White & Ballentine

Allan Timmermann

UCSD ; Centre for Economic Policy Research (CEPR)

Halbert L. White, Jr.

University of California, San Diego (UCSD) - Department of Economics

Date Written: November 2001

Abstract

Data sharing is common practice in forecasting experiments in situations where fresh data samples are difficult or expensive to generate. This means that forecasters often analyze the same data set using a host of different models and sets of explanatory variables. This practice introduces statistical dependencies across forecasting studies that can severely distort statistical inference. Here we examine a new and inexpensive recursive bootstrap procedure that allows forecasters to account explicitly for these dependencies. The procedure allows forecasters to merge empirical evidence and draw inference in the light of previously accumulated results. In an empirical example, we merge results from predictions of daily stock prices based on (1) technical trading rules and (2) calendar rules, demonstrating both the significance of problems arising from data sharing and the simplicity of accounting for data sharing using these new methods.

Keywords: Forecast evaluation, bootstrap, data sharing, calendar effects, technical trading

JEL Classification: C10

Suggested Citation

Sullivan, Ryan M. and Timmermann, Allan and White, Halbert L., Forecast Evaluation with Shared Data Sets (November 2001). Available at SSRN: https://ssrn.com/abstract=292656

Ryan M. Sullivan (Contact Author)

Bates White & Ballentine ( email )

Del Mar, CA
United States

Allan Timmermann

UCSD ( email )

9500 Gilman Drive
La Jolla, CA 92093-0553
United States
858-534-0894 (Phone)

HOME PAGE: http://rady.ucsd.edu/people/faculty/timmermann/

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Halbert L. White

University of California, San Diego (UCSD) - Department of Economics ( email )

9500 Gilman Drive
La Jolla, CA 92093-0508
United States
858-534-3502 (Phone)
858-534-7040 (Fax)

HOME PAGE: http://www.econ.ucsd.edu/~mbacci/white/