Anchoring Vignettes and Response Consistency

33 Pages Posted: 3 Apr 2011

See all articles by Arie Kapteyn

Arie Kapteyn

Dornsife Center for Economic and Social Research - University of Southern California; IZA Institute of Labor Economics; National Bureau of Economic Research (NBER)

James P. Smith

RAND Corporation; IZA Institute of Labor Economics

Arthur van Soest

Tilburg University; Netspar; RAND Corporation; Institute for the Study of Labor (IZA)

Hana Vonkova

Charles University in Prague, Faculty of Education; Tilburg University

Date Written: February 2, 2011

Abstract

The use of anchoring vignettes to correct for differential item functioning rests upon two identifying assumptions: vignette equivalence and response consistency. To test the second assumption the authors conduct an experiment in which respondents in an Internet panel are asked to both describe their health in a number of domains and rate their health in these domains. In a subsequent interview respondents are shown vignettes that are in fact descriptions of their own health. Under response consistency and some auxiliary assumptions with regard to the validity of the experiment, there should be no systematic differences between the evaluation of these vignettes in the second interview and the self-evaluations in the first interview. They analyze data for five health domains: sleep, mobility, concentration, breathing and affect. Although descriptively the vignettes and the self-evaluations are similar for a number of domains, their nonparametric analysis suggests that response consistency is satisfied for the domain of sleep, while it indicates rejection of either the auxiliary assumptions or response consistency for the other domains of health. Parametric analysis suggests that the auxiliary assumptions may be most problematic. The analysis points at the need for a systematic experimental approach to the design of anchoring vignettes before using them in practice.

Suggested Citation

Kapteyn, Arie and Smith, James P. and van Soest, Arthur H. O. and van Soest, Arthur H. O. and Vonkova, Hana, Anchoring Vignettes and Response Consistency (February 2, 2011). RAND Working Paper Series WR-840, Available at SSRN: https://ssrn.com/abstract=1799563 or http://dx.doi.org/10.2139/ssrn.1799563

Arie Kapteyn (Contact Author)

Dornsife Center for Economic and Social Research - University of Southern California ( email )

635 Downey Way
Los Angeles, CA 90089-3332
United States
310-448-5383 (Phone)

IZA Institute of Labor Economics

P.O. Box 7240
Bonn, D-53072
Germany

National Bureau of Economic Research (NBER) ( email )

1050 Massachusetts Avenue
Cambridge, MA 02138
United States

James P. Smith

RAND Corporation ( email )

P.O. Box 2138
1776 Main Street
Santa Monica, CA 90407-2138
United States

IZA Institute of Labor Economics ( email )

P.O. Box 7240
Bonn, D-53072
Germany

Arthur H. O. van Soest

Tilburg University ( email )

P.O. Box 90153
Tilburg, DC Noord-Brabant 5000 LE
Netherlands

Netspar

P.O. Box 90153
Tilburg, 5000 LE
Netherlands

RAND Corporation ( email )

P.O. Box 2138
1776 Main Street
Santa Monica, CA 90407-2138
United States

Institute for the Study of Labor (IZA)

P.O. Box 7240
Bonn, D-53072
Germany

Hana Vonkova

Charles University in Prague, Faculty of Education ( email )

M.D. Rettigove 4
Praha 1, 11639
Czech Republic

Tilburg University ( email )

P.O. Box 90153
Tilburg, DC Noord-Brabant 5000 LE
Netherlands

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
191
Abstract Views
1,817
Rank
286,427
PlumX Metrics