Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging

Abstract

Accurate measurements of probabilistic beliefs have become increasingly important both in practice and in academia. Introduced by statisticians in the 1950s to promote truthful reports in simple environments, Proper Scoring Rules (PSR) are now arguably the most popular incentivized mechanisms to elicit an agent's beliefs. This paper generalizes the analysis of PSR to richer environments relevant to economists. More specifically, we combine theory and experiment to study how beliefs reported with a PSR may be biased when i) the PSR payments are increased, ii) the agent has a financial stake in the event she is predicting, and iii) the agent can hedge her prediction by taking an additional action. Our results reveal complex distortions of reported beliefs, thereby raising concerns about the ability of PSR to recover truthful beliefs in general economic environments.

JEL codes

Replaces

Olivier Armantier, and Nicolas Treich, “Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging”, TSE Working Paper, n. 10-156, April 15, 2010.

Olivier Armantier, and Nicolas Treich, “Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging”, TSE Working Paper, n. 10-213, December 2010.

Reference

Olivier Armantier, and Nicolas Treich, “Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging”, European Economic Review, vol. 62, 2013, pp. 17–40.