Improving the Statistical Power and Reliability of Research Using Amazon Mechanical Turk
41 Pages Posted: 28 Feb 2017 Last revised: 11 Jan 2021
Date Written: December 29, 2020
Abstract
Amazon Mechanical Turk (MTurk) is an increasingly popular source of experimental subjects for accounting research due to its convenience and low cost (relative to traditional laboratories). However, MTurk presents challenges related to statistical power and reliability. These challenges are not unique to MTurk, but are more prevalent than in research conducted with other participant pools. In this paper I discuss several reasons why research conducted with MTurk may face additional power and reliability challenges. I then present suggestions for dealing with these challenges, taking advantage of the comparative strengths of MTurk. The discussion should be of interest to PhD students and other researchers considering using MTurk or other online platforms as a source of experimental participants as well as to reviewers and editors who are considering quality control standards for research conducted with this participant pool.
Keywords: Amazon Mechanical Turk; MTurk; AMT; Effect Size; Sample Size; Screening
JEL Classification: M40, M41, M42, C18, C90, C91
Suggested Citation: Suggested Citation