The Limitations of Experimental Design: A Case Study Involving Monetary Incentive Effects in Laboratory Markets

24 Pages Posted: 28 Feb 2004

Date Written: February 2004

Abstract

We replicate an influential study of monetary incentive effects by Jamal and Sunder (1991) to illustrate the difficulties of drawing causal inferences from a treatment manipulation when other features of the experimental design vary simultaneously. We first show that the Jamal and Sunder (1991) conclusions hinge on one of their laboratory market sessions, conducted only within their fixed-pay condition, that is characterized by a thin market and asymmetric supply and demand curves. When we replicate this structure multiple times under both fixed pay and pay tied to performance, our findings do not support Jamal and Sunder's (1991) conclusion about the incremental effects of performance-based compensation, suggesting that other features varied in that study likely account for their observed difference. Our ceteris paribus replication leaves us unable to offer any generalized conclusions about the effects of monetary incentives in other market structures, but the broader point is to illustrate that experimental designs that attempt to generalize effects by varying multiple features simultaneously can jeopardize the ability to draw causal inferences about the primary treatment manipulation.

Keywords: experimental design, monetary incentives, market power

JEL Classification: C90, C91, C92

Suggested Citation

Kachelmeier, Steven J. and Towry, Kristy L., The Limitations of Experimental Design: A Case Study Involving Monetary Incentive Effects in Laboratory Markets (February 2004). Available at SSRN: https://ssrn.com/abstract=508743 or http://dx.doi.org/10.2139/ssrn.508743

Steven J. Kachelmeier (Contact Author)

University of Texas at Austin ( email )

Department of Accounting
2110 Speedway, Mail Stop B6400
Austin, TX 78712
United States
512-471-3517 (Phone)
512-471-3904 (Fax)

Kristy L. Towry

Emory University ( email )

Goizueta Business School
1300 Clifton Road
Atlanta, GA 30322
United States
404-727-4895 (Phone)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
332
Abstract Views
5,946
Rank
166,135
PlumX Metrics