Methodological uncertainty in portfolio sorts

Abstract

Systematically studying methodological variation in portfolio sorts reveals four key insights: (1) The average monthly non-standard error is 0.19% and exceeds standard errors. Despite this considerable variation, the majority of estimated premia are robust regarding their sign, statistical significance, and monotonicity. This alleviates concerns about replicability. (2) Decisions such as excluding firms with negative earnings or the information lag have an impact comparable to size-related choices. (3) Methodological choices induce not just orthogonal noise but add predictably non-zero returns. (4) To incorporate methodological uncertainty, we propose a two-step protocol adaptable to economic motivations, for which we provide an open-source tool.

Keywords: Non-standard errors, portfolio sorts, data mining, risk factors, anomalies

JEL Classification: C58, G10, G11, G12, G14