Finance is Not Excused: Why Finance Should Not Flout Basic Principles of Statistics

Abstract

Several features of financial research make it particularly prone to the occurrence of false discoveries. First, the probability of finding a positive (profitable investment strategy) is very low, due to intense competition. Second, true findings are mostly short-lived, as a result of the non-stationary nature of financial systems. Third, unlike in the natural sciences, it is rarely possible to verify statistical findings through controlled experiments. Finance’s inability to conduct controlled experiments makes it virtually impossible to debunk a false claim. One would hope that, in such a field, researchers would be particularly careful when conducting statistical inference. Sadly, the opposite is true.

Tenure-seeking researchers publish thousands of academic articles that promote dubious investment strategies, without controlling for multiple testing. Some of those articles are written for, funded, or promoted by investment firms with a commercial interest. As a consequence, today’s academic finance exhibits some resemblance with medicine’s predicament during the 1950-2000 period, when Big Tobacco paid for thousands of studies in support of their bottom line. Unlike finance, medical journals today impose strict controls for multiple testing. Academic finance’s denial of its replication crisis risks its branding as a pseudoscience.

Keywords: Multiple testing, selection bias, publication bias, false discovery rate, true positive rate, deflated Sharpe ratio

JEL Classification: G0, G1, G2, G15, G24, E44