P-Curve Won’t Do Your Laundry, But It Will Distinguish Replicable from Non-Replicable Findings in Observational Research: Comment on Bruns & Ioannidis (2016)

Forthcoming, Plos ONE

8 Pages Posted: 14 Dec 2018

See all articles by Uri Simonsohn

Uri Simonsohn

ESADE Business School; Ramon Llull University - ESADE Business School; ESADE Business School

Leif D. Nelson

University of California, Berkeley - Haas School of Business

Joseph P. Simmons

University of Pennsylvania - The Wharton School

Date Written: November 22, 2018

Abstract

P-curve, the distribution of significant p-values, can be analyzed to assess if the findings have evidential value, whether p-hacking and file-drawering can be ruled out as the sole explanations for them. Bruns and Ioannidis (2016) have proposed p-curve cannot examine evidential value with observational data. Their discussion confuses false-positive findings with confounded ones, failing to distinguish correlation from causation. We demonstrate this important distinction by showing that a confounded but real, hence replicable association, gun ownership and number of sexual partners, leads to a right-skewed p-curve, while a false-positive one, respondent ID number and trust in the supreme court, leads to a flat p-curve. P-curve can distinguish between replicable and non-replicable findings. The observational nature of the data is not consequential.

Suggested Citation

Simonsohn, Uri and Nelson, Leif D. and Simmons, Joseph P., P-Curve Won’t Do Your Laundry, But It Will Distinguish Replicable from Non-Replicable Findings in Observational Research: Comment on Bruns & Ioannidis (2016) (November 22, 2018). Forthcoming, Plos ONE, Available at SSRN: https://ssrn.com/abstract=3289096

Uri Simonsohn (Contact Author)

ESADE Business School ( email )

Av. de Pedralbes, 60-62
Barcelona, 08034
Spain

Ramon Llull University - ESADE Business School ( email )

Avinguda de la Torre Blanca, 59
Sant Cugat del Vallès, 08172
Spain

HOME PAGE: http://urisohn.com

ESADE Business School ( email )

Av. de Pedralbes, 60-62
Barcelona, 08034
Spain

Leif D. Nelson

University of California, Berkeley - Haas School of Business ( email )

545 Student Services Building, #1900
2220 Piedmont Avenue
Berkeley, CA 94720
United States

Joseph P. Simmons

University of Pennsylvania - The Wharton School ( email )

3733 Spruce Street
Philadelphia, PA 19104-6374
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
121
Abstract Views
1,297
Rank
417,254
PlumX Metrics