Fraud in Online Surveys: Evidence from a Nonprobability, Subpopulation Sample

Journal of Experimental Political Science (Forthcoming)

17 Pages Posted: 17 Jan 2022 Last revised: 7 Apr 2022

See all articles by Andrew M. Bell

Andrew M. Bell

Indiana University- Bloomington; University College London

Thomas Gift

University College London

Date Written: October 5, 2021

Abstract

We hired a well-known market research firm whose surveys have been published in leading political science journals, including JEPS. Based on a set of rigorous "screeners,'' we detected what appears to be exceedingly high rates of identity falsification: Over 81 percent of respondents seemed to misrepresent their credentials to gain access to the survey and earn compensation. Similarly high rates of presumptive character falsification were present in panels from multiple sub-vendors procured by the firm. Moreover, we found additional, serious irregularities embedded in the data, including evidence of respondents using deliberate strategies to detect and circumvent one of our screeners, as well as pervasive, observable patterns reflecting that the survey had been taken repeatedly by a respondent or collection of respondents. This evidence offers particular reasons to be concerned about the quality of digital, non-probability, subpopulation samples, and calls for further, systematic research.

Keywords: surveys, fraud, methodology, samples, online surveys, military

Suggested Citation

Bell, Andrew M. and Gift, Thomas, Fraud in Online Surveys: Evidence from a Nonprobability, Subpopulation Sample (October 5, 2021). Journal of Experimental Political Science (Forthcoming), Available at SSRN: https://ssrn.com/abstract=3957523 or http://dx.doi.org/10.2139/ssrn.3957523

Andrew M. Bell (Contact Author)

Indiana University- Bloomington ( email )

Bloomington, IN
United States
9192369583 (Phone)

HOME PAGE: http://https://andrewmbell.com/

University College London ( email )

Gower Street
London, WC1E 6BT
United Kingdom

Thomas Gift

University College London ( email )

Gower Street
London, WC1E 6BT
United Kingdom

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
55
Abstract Views
280
Rank
670,186
PlumX Metrics