Probability Elicitation, Scoring Rules, and Competition Among Forecasters

23 Pages Posted: 5 Feb 2007

See all articles by Kenneth C. Lichtendahl

Kenneth C. Lichtendahl

University of Virginia - Darden School of Business

Robert L. Winkler

Duke University - Fuqua School of Business

Date Written: January 21, 2007

Abstract

Probability forecasters who are rewarded via a proper scoring rule may care not only about the score, but also about their performance relative to other forecasters. We model this type of preference and show that a competitive forecaster who wants to do better than another forecaster typically should report more extreme probabilities, exaggerating toward zero or one. We consider a competitive forecaster's best response to truthful reporting and also investigate equilibrium reporting functions in the case where another forecaster also cares about relative performance. We show how a decision maker can revise probabilities of an event after receiving reported probabilities from competitive forecasters and note that the strategy of exaggerating probabilities can make well-calibrated forecasters (and a decision maker who takes their reported probabilities at face value) appear to be overconfident. However, a decision maker who adjusts appropriately for the misrepresentation of probabilities by one or more forecasters can still be well-calibrated. Finally, to try to overcome the forecasters' competitive instincts and induce cooperative behavior, we develop the notion of joint scoring rules based on business sharing and show that these scoring rules are strictly proper.

Keywords: probability elicitation; scoring rules; forecasting competitions; probability forecasts; truthful revelation; overconfidence bias

Suggested Citation

Lichtendahl, Kenneth C. and Winkler, Robert L., Probability Elicitation, Scoring Rules, and Competition Among Forecasters (January 21, 2007). Darden Business School Working Paper No. 07-05, Available at SSRN: https://ssrn.com/abstract=961001 or http://dx.doi.org/10.2139/ssrn.961001

Kenneth C. Lichtendahl (Contact Author)

University of Virginia - Darden School of Business ( email )

P.O. Box 6550
Charlottesville, VA 22906-6550
United States

Robert L. Winkler

Duke University - Fuqua School of Business ( email )

Box 90120
Durham, NC 27708-0120
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
273
Abstract Views
1,772
Rank
203,702
PlumX Metrics