Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments

66 Pages Posted: 16 Oct 2017 Last revised: 19 May 2023

See all articles by Daniel J. Benjamin

Daniel J. Benjamin

USC, Center for Economic and Social Research (CESR); Anderson School of Management; National Bureau of Economic Research (NBER); Human Genetics Department, David Geffen School of Medicine

Don Moore

University of California, Berkeley - Haas School of Business

Matthew Rabin

Harvard University - Department of Economics

Multiple version iconThere are 3 versions of this paper

Date Written: October 2017

Abstract

This paper describes results of a pair of incentivized experiments on biases in judgments about random samples. Consistent with the Law of Small Numbers (LSN), participants exaggerated the likelihood that short sequences and random subsets of coin flips would be balanced between heads and tails. Consistent with the Non-Belief in the Law of Large Numbers (NBLLN), participants underestimated the likelihood that large samples would be close to 50% heads. However, we identify some shortcomings of existing models of LSN, and we find that NBLLN may not be as stable as previous studies suggest. We also find evidence for exact representativeness (ER), whereby people tend to exaggerate the likelihood that samples will (nearly) exactly mirror the underlying odds, as an additional bias beyond LSN. Our within-subject design of asking many different questions about the same data lets us disentangle the biases from possible rational alternative interpretations by showing that the biases lead to inconsistency in answers. Our design also centers on identifying and controlling for bin effects, whereby the probability assigned to outcomes systematically depends on the categories used to elicit beliefs in a way predicted by support theory. The bin effects are large and systematic and affect some results, but we find LSN, NBLLN, and ER even after controlling for them.

Suggested Citation

Benjamin, Daniel J. and Benjamin, Daniel J. and Moore, Don and Rabin, Matthew, Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments (October 2017). NBER Working Paper No. w23927, Available at SSRN: https://ssrn.com/abstract=3053717

Daniel J. Benjamin (Contact Author)

USC, Center for Economic and Social Research (CESR) ( email )

635 Downey Way
Los Angeles, CA 90089-3332
United States

Anderson School of Management ( email )

110 Westwood Plaza
Los Angeles, CA 90095-1481
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Human Genetics Department, David Geffen School of Medicine ( email )

695 Charles E. Young Drive South
Los Angeles, CA 90095-7088
United States

Don Moore

University of California, Berkeley - Haas School of Business ( email )

545 Student Services Building, #1900
2220 Piedmont Avenue
Berkeley, CA 94720
United States

Matthew Rabin

Harvard University - Department of Economics ( email )

Littauer Center
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
27
Abstract Views
407
PlumX Metrics