Evaluation of Development Programs: Using Regressions to Assess the Impact of Complex Interventions

Abstract

There is a growing interest in extending project evaluation methods to the evaluation of programs: complex interventions involving multiple activities. In general a program evaluation cannot be based on separate evaluations of its components since interactions between the activities are likely to be important. We propose a measure of program impact, the total program effect (TPE), which is an extension of the average treatment effect on the treated (ATET). Regression techniques can be applied to observational data from a representative sample to estimate the TPE for complex interventions in the presence of selection effects and treatment heterogeneity. As an example we present an estimate of the TPE for a rural water supply and sanitation program in Mozambique. Estimating the TPE from randomized controlled trials would appear to be an alternative; however, the scope for using RCTs in this context is limited.

Keywords: program evaluation, randomized controlled trials, policy evaluation, treatment heterogeneity, budget support, sector-wide programs, aid effectiveness

JEL Classification: C21, C33, O22