Exploring the Usefulness of a Non-Random Holdout Sample for Model Validation: Welfare Effects on Female Behavior

85 Pages Posted: 10 Mar 2006

See all articles by Michael P. Keane

Michael P. Keane

Arizona State University (ASU) - Economics Department; University of Technology, Sydney (Visiting July 2006-Present)

Kenneth I. Wolpin

University of Pennsylvania - Department of Economics

Date Written: May 2005

Abstract

Opportunities for external validation of behavioral models in the social sciences that are based on randomized social experiments or on large regime shifts, that can be treated as experiments for the purpose of model validation, are extremely rare. In this paper, we consider an alternative approach, namely mimicking the essential element of regime change by non-randomly holding out from estimation a portion of the sample that faces a significantly different policy regime. The non-random holdout sample is used for model validation/selection. We illustrate the non-random holdout sample approach to model validation in the context of a model of welfare program participation. The policy heterogeneity that we exploit to generate a non-random holdout sample takes advantage of the wide variation across states that has existed in welfare policy.

Keywords: Model validation, Hold-out sample, Public welfare

JEL Classification: C52, C53, J1, J2

Suggested Citation

Keane, Michael P. and Wolpin, Kenneth I., Exploring the Usefulness of a Non-Random Holdout Sample for Model Validation: Welfare Effects on Female Behavior (May 2005). PIER Working Paper No. 06-006, Available at SSRN: https://ssrn.com/abstract=889567 or http://dx.doi.org/10.2139/ssrn.889567

Michael P. Keane

Arizona State University (ASU) - Economics Department ( email )

Tempe, AZ 85287-3806
United States

University of Technology, Sydney (Visiting July 2006-Present)

PO Box 123 Broadway
NSW 2007
Australia
480-965-1053 (Phone)
480-965-0748 (Fax)

Kenneth I. Wolpin (Contact Author)

University of Pennsylvania - Department of Economics ( email )

Ronald O. Perelman Center for Political Science
133 South 36th Street
Philadelphia, PA 19104-6297
United States
215-898-7708 (Phone)
215-573-2057 (Fax)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
115
Abstract Views
1,802
Rank
433,058
PlumX Metrics