Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings

67 Pages Posted: 22 Oct 2019

See all articles by Bharat Chandar

Bharat Chandar

Stanford Graduate School of Business

Ali Hortaçsu

University of Chicago; National Bureau of Economic Research (NBER)

John A. List

University of Chicago - Department of Economics

Ian Muir

Lyft, Inc.

Jeffrey M. Wooldridge

Michigan State University - Department of Economics

Multiple version iconThere are 2 versions of this paper

Date Written: October 2019

Abstract

Field experiments conducted with the village, city, state, region, or even country as the unit of randomization are becoming commonplace in the social sciences. While convenient, subsequent data analysis may be complicated by the constraint on the number of clusters in treatment and control. Through a battery of Monte Carlo simulations, we examine best practices for estimating unit-level treatment effects in cluster-randomized field experiments, particularly in settings that generate short panel data. In most settings we consider, unit-level estimation with unit fixed effects and cluster-level estimation weighted by the number of units per cluster tend to be robust to potentially problematic features in the data while giving greater statistical power. Using insights from our analysis, we evaluate the effect of a unique field experiment: a nationwide tipping field experiment across markets on the Uber app. Beyond the import of showing how tipping affects aggregate market outcomes, we provide several insights on aspects of generating and analyzing cluster-randomized experimental data when there are constraints on the number of experimental units in treatment and control.

JEL Classification: C23, C33, C5, C9, C91, C92, C93, D47

Suggested Citation

Chandar, Bharat and Hortaçsu, Ali and List, John A. and Muir, Ian and Wooldridge, Jeffrey M., Design and Analysis of Cluster-Randomized Field Experiments in Panel Data Settings (October 2019). University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2019-129, Available at SSRN: https://ssrn.com/abstract=3473409 or http://dx.doi.org/10.2139/ssrn.3473409

Bharat Chandar

Stanford Graduate School of Business ( email )

Stanford, CA
United States

Ali Hortaçsu

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

John A. List (Contact Author)

University of Chicago - Department of Economics ( email )

1126 East 59th Street
Chicago, IL 60637
United States

Ian Muir

Lyft, Inc. ( email )

San Francisco, CA

Jeffrey M. Wooldridge

Michigan State University - Department of Economics ( email )

#211 Marshall Hall
East Lansing, MI 48824-1038
United States
517+353-5972 (Phone)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
73
Abstract Views
835
Rank
486,005
PlumX Metrics