Uncovering the Iceberg from Its Tip: A Model of Publication Bias and p-Hacking

Abstract

Harvey, Liu, and Zhu (2016) argue that a large proportion of published asset-pricing factors are likely false. Researchers may try many variables and report only the significant ones, so-called p-hacking. Some recent work challenges the prevalence of p-hacking and argues that the amount of shrinkage necessary for reported results is trivial. We present a model where there are true anomalies and false anomalies. Our model does a good job of fitting the observed population. Our evidence is consistent with the idea that a large proportion of anomalies are false and reinforces the need to raise the thresholds for statistical significance.

Keywords: p-hacking, Data mining, Anomalies, Simulations, Publication bias, Multiple testing

JEL Classification: C12, G10, G12