Leveraging Lotteries for School Value-Added: Testing and Estimation

69 Pages Posted: 23 Nov 2015 Last revised: 5 Jun 2022

See all articles by Joshua D. Angrist

Joshua D. Angrist

Massachusetts Institute of Technology (MIT) - Department of Economics; National Bureau of Economic Research (NBER); IZA Institute of Labor Economics

Peter Hull

Microsoft Corporation - Microsoft Research New England; University of Chicago

Parag A. Pathak

Massachusetts Institute of Technology (MIT) - Department of Economics

Christopher Walters

University of California, Berkeley - Department of Economics

Date Written: November 2015

Abstract

Conventional value-added models (VAMs) compare average test scores across schools after regression-adjusting for students’ demographic characteristics and previous scores. This paper tests for VAM bias using a procedure that asks whether VAM estimates accurately predict the achievement consequences of random assignment to specific schools. Test results from admissions lotteries in Boston suggest conventional VAM estimates are biased, which motivates the development of a hierarchical model describing the joint distribution of school value-added, bias, and lottery compliance. We use this model to assess the substantive importance of bias in conventional VAM estimates and to construct hybrid value-added estimates that optimally combine ordinary least squares and lottery-based instrumental variables estimates of VAM parameters. The hybrid estimation strategy provides a general recipe for combining non-experimental and quasi-experimental estimates. While still biased, hybrid school value-added estimates have lower mean squared error than conventional VAM estimates. Simulations calibrated to the Boston data show that, bias notwithstanding, policy decisions based on conventional VAMs are likely to generate substantial achievement gains. Hybrid estimates that incorporate lotteries yield further gains.

Suggested Citation

Angrist, Joshua and Hull, Peter and Pathak, Parag A. and Walters, Christopher, Leveraging Lotteries for School Value-Added: Testing and Estimation (November 2015). NBER Working Paper No. w21748, Available at SSRN: https://ssrn.com/abstract=2694415

Joshua Angrist (Contact Author)

Massachusetts Institute of Technology (MIT) - Department of Economics ( email )

50 Memorial Drive
E52-353
Cambridge, MA 02142
United States
617-253-8909 (Phone)
617-253-1330 (Fax)

National Bureau of Economic Research (NBER)

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

IZA Institute of Labor Economics

P.O. Box 7240
Bonn, D-53072
Germany

Peter Hull

Microsoft Corporation - Microsoft Research New England ( email )

One Memorial Drive, 14th Floor
Cambridge, MA 02142
United States

HOME PAGE: http://https://www.microsoft.com/en-us/research/people/pehull/

University of Chicago ( email )

1101 East 58th Street
Chicago, IL 60637
United States

HOME PAGE: http://about.peterhull.net/

Parag A. Pathak

Massachusetts Institute of Technology (MIT) - Department of Economics ( email )

50 Memorial Drive
E52-391
Cambridge, MA 02142
United States

Christopher Walters

University of California, Berkeley - Department of Economics ( email )

549 Evans Hall #3880
Berkeley, CA 94720-3880
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
33
Abstract Views
567
PlumX Metrics