Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study

82 Pages Posted: 8 Sep 2015 Last revised: 17 Apr 2023

See all articles by Ben Gillen

Ben Gillen

California Institute of Technology - Division of the Humanities and Social Sciences; Claremont Colleges, Claremont McKenna College, Robert Day School of Economics and Finance, Students

Erik Snowberg

California Institute of Technology - Division of the Humanities and Social Sciences; National Bureau of Economic Research (NBER)

Leeat Yariv

Princeton University; Centre for Economic Policy Research (CEPR); National Bureau of Economic Research (NBER)

Date Written: September 2015

Abstract

Measurement error is ubiquitous in experimental work. It leads to imperfect statistical controls, attenuated estimated effects of elicited behaviors, and biased correlations between characteristics. We develop simple statistical techniques for dealing with experimental measurement error. These techniques are applied to data from the Caltech Cohort Study, which conducts repeated incentivized surveys of the Caltech student body. We illustrate the impact of measurement error by replicating three classic experiments, and showing that results change substantially when measurement error is taken into account. Collectively, these results show that failing to properly account for measurement error may cause a field-wide bias: it may lead scholars to identify "new" effects and phenomena that are actually similar to those previously documented.

Suggested Citation

Gillen, Ben and Gillen, Ben and Snowberg, Erik and Yariv, Leeat, Experimenting with Measurement Error: Techniques with Applications to the Caltech Cohort Study (September 2015). NBER Working Paper No. w21517, Available at SSRN: https://ssrn.com/abstract=2656900

Ben Gillen (Contact Author)

California Institute of Technology - Division of the Humanities and Social Sciences ( email )

1200 East California Blvd.
Pasadena, CA 91125
United States

Claremont Colleges, Claremont McKenna College, Robert Day School of Economics and Finance, Students ( email )

500 E. Ninth St.
Claremont, CA 91711-6420
United States

Erik Snowberg

California Institute of Technology - Division of the Humanities and Social Sciences ( email )

1200 East California Blvd.
Pasadena, CA 91125
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Leeat Yariv

Princeton University ( email )

Princeton, NJ 08544-1021
United States

Centre for Economic Policy Research (CEPR)

London
United Kingdom

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
109
Abstract Views
721
Rank
450,735
PlumX Metrics