Do People Mean What They Say? Implications for Subjective Survey Data

Abstract

Many surveys contain a wealth of subjective questions that are at first glance rather exciting. Examples include "How important is leisure time to you?" "How satisfied are you with yourself?"; or "How satisfied are you with your work?" Yet despite easy availability, this is one data source that economists rarely use. In fact, the unwillingness to rely on such questions marks an important divide between economists and other social scientists.

This neglect does not come from disinterest. Most economists would probably agree that the variables these questions attempt to uncover are interesting and important. But they doubt whether these questions elicit meaningful answers. These doubts are, however, based on a priori skepticism rather than on evidence. This ignores a large body of experimental and empirical work that has investigated the meaningfulness of answers to these questions. Our primary objective in this paper is to summarize this literature for an audience of economists. Thereby turning a vague implicit distrust into an explicit position grounded in facts. Having summarized the findings, we integrate them into a measurement error framework so as to understand what they imply for empirical research relying on subjective data. Finally, in order to calibrate the extent of the measurement error problem, we perform some simple empirical work using specific subjective questions.

JEL Classification: C8, D0, J0