Targeted Testing for Bias in Order Assignment, with an Application to Texas Election Ballots

Abstract

Statistical methods are developed for assessing the likelihood of prejudicial bias in agent-assigned permutations, such as the ordering of candidates on an election ballot. The null hypothesis of an unbiased order assignment is represented by several forms of probabilistic exchangeability of the random orderings, while bias is represented either by compatibility with an assumed ranking of the items with respect to a hypothesized preference criterion (PC) or by linear concordance with assumed scores of the items on a PC scale. A power analysis indicates the superiority of these methods to an neutral alternative when appropriate a priori information is available; their usefulness is affirmed in an application to the ordering of candidates on 2014 Texas Republican primary election ballots. Significant evidence of bias is found in three of the five races studied, a finding that does not obtain using currently available tests.

Keywords: ballot-order effect, primacy effect, bias, ordering, ranking, hypothesis testing

JEL Classification: C12, C49