Uncovering Bias in Order Assignment
37 Pages Posted: 6 Oct 2020 Last revised: 28 Sep 2021
Date Written: September 22, 2020
Abstract
To mitigate sequencing effects in decision-making, which are widespread, many real-life situations require a set of items to be considered in a random order. When such orderings are repeated, one can test whether randomization indeed obtains, or whether some individuals manipulate them in order to achieve a favorable result. However, the literature offers limited guidance on how to conduct these tests. This paper articulates the key features of this problem and presents three general tests that require no a priori information from the analyst. These methods are used to analyze the order in which lottery numbers are drawn in Powerball, the order in which contestants perform on American Idol, and the order of candidates on primary election ballots in Texas and West Virginia. In this last application, multiple deviations from full randomization are detected, with potentially serious political and legal consequences.
Keywords: ordering; sequencing; hypothesis testing; randomization; ballot order
JEL Classification: C12, D91
Suggested Citation: Suggested Citation