Optimal Experimental Design for Staggered Rollouts

111 Pages Posted: 20 Nov 2019 Last revised: 30 Mar 2023

See all articles by Ruoxuan Xiong

Ruoxuan Xiong

Emory University

Susan Athey

Stanford Graduate School of Business

Mohsen Bayati

Stanford Graduate School of Business

Guido W. Imbens

Stanford Graduate School of Business

Date Written: November 9, 2019

Abstract

In this paper, we study the design and analysis of experiments conducted on a set of units over multiple time periods where the starting time of the treatment may vary by unit. The design problem involves selecting an initial treatment time for each unit in order to most precisely estimate both the instantaneous and cumulative effects of the treatment. We first consider non-adaptive experiments, where all treatment assignment decisions are made prior to the start of the experiment. For this case, we show that the optimization problem is generally NP-hard and we propose a near-optimal solution. Under this solution the fraction entering treatment each period is initially low, then high, and finally low again. Next, we study an adaptive experimental design problem, where both the decision to continue the experiment and treatment assignment decisions are updated after each period's data is collected. For the adaptive case we propose a new algorithm, the Precision-Guided Adaptive Experiment (PGAE) algorithm, that addresses the challenges at both the design stage and at the stage of estimating treatment effects, ensuring valid post-experiment inference accounting for the adaptive nature of the design. Using realistic settings, we demonstrate that our proposed solutions can reduce the opportunity cost of the experiments by over 50\%, compared to static design benchmarks.

Keywords: Adaptive Experiments, Treatment Effect Estimation, Carryover Effects, Panel Data, Dynamic programming

Suggested Citation

Xiong, Ruoxuan and Carleton Athey, Susan and Bayati, Mohsen and Imbens, Guido W., Optimal Experimental Design for Staggered Rollouts (November 9, 2019). Available at SSRN: https://ssrn.com/abstract=3483934 or http://dx.doi.org/10.2139/ssrn.3483934

Ruoxuan Xiong (Contact Author)

Emory University ( email )

36 Eagle Row
Atlanta, GA 30322-0001
United States
4707273668 (Phone)

Susan Carleton Athey

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Mohsen Bayati

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

HOME PAGE: http://web.stanford.edu/~bayati/

Guido W. Imbens

Stanford Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
691
Abstract Views
2,740
Rank
69,289
PlumX Metrics