Measuring the Impacts of Teachers: Response to Rothstein (2014)

54 Pages Posted: 18 Aug 2015

See all articles by Raj Chetty

Raj Chetty

Harvard University

John Friedman

Brown University

Jonah E. Rockoff

Columbia University - Columbia Business School, Finance; National Bureau of Economic Research (NBER)

Date Written: August 2015

Abstract

Using data from North Carolina, Jesse Rothstein (2014) presents a comprehensive replication of Chetty, Friedman, and Rockoff's [CFR] (2014a,b) results on teachers' impacts. In addition, Rothstein presents new evidence that he argues raises concerns about three aspects of CFR's methods and identification assumptions: their treatment of missing data, the validity of their quasi-experimental design, and their method of controlling for observables when estimating teachers' long-term effects. In this paper, we show that Rothstein's methodological critiques are not valid by replicating his new empirical findings using simulated data in which none of CFR's identification assumptions are violated. We also present supplementary empirical evidence from our data supporting the assumptions required for CFR's original analyses. Together, these results show that: (1) Rothstein's technique for imputing teacher VA for teachers with missing data generates bias, while subsamples with no missing data yield estimates of forecast bias similar to CFR's baseline results; (2) his proposed prior score \placebo test" rejects valid quasiexperimental research designs, and the correlation between changes in prior test scores and current teacher value-added he documents is an artifact of estimating teacher value-added using prior test score data; and (3) his method of controlling for covariates yields inconsistent estimates of teachers' long-term effects, while quasi-experimental designs that do not rely on controls for observables yield estimates of teachers' long-term impacts similar to CFR's baseline results. We conclude that Rothstein's important replication study is entirely consistent with – and in fact reinforces – CFR's methods and results. Our conclusions match those of Bacher-Hicks, Kane, and Staiger (2014), who replicate both CFR's results and Rothstein's findings using data from Los Angeles and also conclude that Rothstein's results raise no concerns about CFR's analysis.

Keywords: education policy, teacher effects, value-added models

JEL Classification: H52, I28, J01

Suggested Citation

Chetty, Raj and Friedman, John and Rockoff, Jonah E., Measuring the Impacts of Teachers: Response to Rothstein (2014) (August 2015). CEPR Discussion Paper No. DP10768, Available at SSRN: https://ssrn.com/abstract=2645721

Raj Chetty (Contact Author)

Harvard University ( email )

1875 Cambridge Street
Cambridge, MA 02138
United States

John Friedman

Brown University ( email )

Box 1860
Providence, RI 02912
United States

Jonah E. Rockoff

Columbia University - Columbia Business School, Finance ( email )

3022 Broadway
New York, NY 10027
United States

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
1
Abstract Views
744
PlumX Metrics