Explaining Apparently Inaccurate Self-assessments of Relative Performance: A Replication and Adaptation of 'Overconfident: Do you put your money on it?' by Hoelzl and Rustichini (2005)

TL;DR

This study replicates and adapts a classic experiment on overconfidence, revealing that participants tend to prefer performance-based rewards, challenging previous findings of underconfidence, and emphasizing the influence of normative and control factors.

Contribution

It introduces a modified lottery mechanism to better match performance incentives and reassesses the nature of self-assessed relative performance in overconfidence research.

Findings

Majority prefer performance-based bonuses regardless of lottery design

Expected performance and group estimations predict voting behavior

Social comparison and risk attitudes are not significant factors

Abstract

This study replicates and adapts the experiment of Hoelzl and Rustichini (2005), which examined overplacement, i.e., overconfidence in relative self-assessments, by analyzing individuals' voting preferences between a performance-based and a lottery-based bonus payment mechanism. The original study found underplacement - the majority of their sample apparently expected to perform worse than others - in difficult tasks with monetary incentives, contradicting the widely held assumption of a general human tendency toward overconfidence. This paper challenges the comparability of the two payment schemes, arguing that differences in outcome structures and non-monetary motives may have influenced participants' choices beyond misconfidence. In an online replication, a fixed-outcome distribution lottery mechanism with interdependent success probabilities and no variance in the number of winners - designed to better align with the performance-based payment scheme - is compared against the probabilistic-outcome distribution lottery used in the original study, which features an independent success probability and a variable number of winners. The results align more closely with traditional overplacement patterns than underplacement, as nearly three-fourths of participants prefer the performance-based option regardless of lottery design. Key predictors of voting behavior include expected performance, group performance estimations, and sample question outcomes, while factors such as social comparison tendencies and risk attitudes play no significant role. Self-reported voting rationales highlight the influence of normative beliefs, control preferences, and feedback signals beyond confidence. These results contribute to methodological discussions in overconfidence research by reassessing choice-based overconfidence measures and exploring alternative explanations for observed misplacement effects.