An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA
Policy Research Working Papers
While providing the most reliable method of evaluating social programs, randomized experiments in developing and developed countries alike are accompanied by political risks and ethical issues that jeopardize the chances of adopting them. In this paper we use a unique data set from rural Mexico collected for the purposes of evaluating the impact of the PROGRESA poverty alleviation program to examine the performance of a quasi-experimental estimator, the Regression Discontinuity Design (RDD).
... ty Design (RDD). Using as a benchmark the impact estimates based on the experimental nature of the sample, we examine how estimates differ when we use the RDD as the estimator for evaluating program impact on two key indicators: child school attendance and child work. Overall the performance of the RDD was remarkably good. The RDD estimates of program impact agreed with the experimental estimates in 10 out of the 12 possible cases. The two cases in which the RDD method failed to reveal any significant program impact on the school attendance of boys and girls were in the first year of the program (round 3). RDD estimates comparable to the experimental estimates were obtained when we used as a comparison group children from non-eligible households in the control localities. 2 and more recently regression discontinuity design. As in social experiments, both of these "quasi-experimental" methods attempt to equalize the selection bias present in treatment and comparison groups so as to yield unbiased estimates of parameters measuring program impact, such as the average treatment effect, the treatment of the treated effect and the local average treatment effect (Blundell and Costa-Diaz, 2002). One critical question associated with these non-experimental approaches is the extent to which they result in impact estimates that are comparable to those that would be obtained with an experimental approach. With these considerations in mind, this paper uses a unique data set from rural Mexico collected for the purposes of evaluating the impact of the PROGRESA poverty alleviation program to examine the performance of the Regression Discontinuity Design (RDD). The PROGRESA program and its impact on a variety of welfare indicators have been extensively evaluated using the experimental design of the sample. 2 In this paper, we evaluate a nonexperimental estimator rather than the impact of the program itself, by examining how different impact estimates would be if one were to use the RDD as the estimator for evaluating program impact on two key indicators: child school attendance and child work. The RDD estimator in the economic literature has been recently used by Van Der KlaauwAngrist and Lavy, 1999, while the identification and estimation of treatment effects with an RD design are discussed in Hahn, Todd and Van der Klaauw, 2001. In the case of PROGRESA the particular method of selecting households eligible for the program benefits provides the basis for the RD design. As discussed in more detail later, all households within a region are ranked media campaign of opposition parties in Mexico prior to the elections of 2000. 2 Skoufias (2001) provides a detailed discussion of PROGRESA, the evaluation design and the experimental estimates impacts of the program.