Dangers of Data-Driven Inference: The Case of Calendar Effects in Stock Returns

UCSD Economics Discussion Paper 98-16

Posted: 20 Aug 1998

See all articles by Ryan Sullivan

Ryan Sullivan

Bates White & Ballentine

Allan Timmermann

UCSD ; Centre for Economic Policy Research (CEPR)

Halbert L. White, Jr.

University of California, San Diego (UCSD) - Department of Economics

Date Written: June 1998

Abstract

Economics is primarily a non-experimental science. Typically, we cannot generate new data sets on which to test hypotheses independently of the data that may have led to a particular theory. The common practice of using the same data set to formulate and test hypotheses introduces data-snooping biases that, if not accounted for, invalidate the assumptions underlying classical statistical inference. A striking example of a data-driven discovery is the presence of calendar effects in stock returns. There appears to be very substantial evidence of systematic abnormal stock returns related to the day of the week, the week of the month, the month of the year, the turn of the month, holidays, and so forth. However, this evidence has largely been considered without accounting for the intensive search preceding it. In this paper we use 100 years of daily data and a new bootstrap procedure that allows us to explicitly measure the distortions in statistical inference induced by data-snooping. We find that although nominal P-values of individual calendar rules are extremely significant, once evaluated in the context of the full universe from which such rules were drawn, calendar effects no longer remain significant.

JEL Classification: G10, G11, G12, C63

Suggested Citation

Sullivan, Ryan M. and Timmermann, Allan and White, Halbert L., Dangers of Data-Driven Inference: The Case of Calendar Effects in Stock Returns (June 1998). UCSD Economics Discussion Paper 98-16, Available at SSRN: https://ssrn.com/abstract=106612

Ryan M. Sullivan

Bates White & Ballentine ( email )

Del Mar, CA
United States

Allan Timmermann (Contact Author)

UCSD ( email )

9500 Gilman Drive
La Jolla, CA 92093-0553
United States
858-534-0894 (Phone)

HOME PAGE: http://rady.ucsd.edu/people/faculty/timmermann/

Centre for Economic Policy Research (CEPR)

London
United Kingdom

Halbert L. White

University of California, San Diego (UCSD) - Department of Economics ( email )

9500 Gilman Drive
La Jolla, CA 92093-0508
United States
858-534-3502 (Phone)
858-534-7040 (Fax)

HOME PAGE: http://www.econ.ucsd.edu/~mbacci/white/

Do you have negative results from your research you’d like to share?

Paper statistics

Abstract Views
1,469
PlumX Metrics