Measuring Choice Difficulty

TL;DR

This paper develops a theoretical framework to analyze how different measures of choice difficulty relate, revealing they are generally unrelated and highlighting conditions where they align, especially in psychophysical tasks.

Contribution

It clarifies the relationships between various measures of choice difficulty and provides conditions under which they coincide, informing interpretation in economics and psychophysics.

Findings

Understanding, choice randomness, and confidence are generally unrelated measures.

In certain conditions, confidence coincides with understanding in psychophysical tasks.

Willingness-to-accept to switch in utils is equivalent to understanding.

Abstract

We provide a theoretical framework to understand how widely used measures of choice difficulty relate. In a binary-option Bayesian expected-utility framework, we show that three measures of difficulty, (i) understanding (ex-ante value), (ii) choice randomness, and (iii) confidence that the chosen option is ex post correct, are, in general, unrelated, and that this result extends to other potential measures like attenuation. We provide intuitive sufficient conditions which align the orders, using both restrictions on Blackwell experiments that capture well known classes (such as logit) and restrictions on payoffs and demonstrate that in psychophysical tasks that pay only for correctness, confidence coincides with understanding. We show willingness-to-accept to switch, when measured in utils, is equivalent to understanding. Our results suggest caution in interpreting measures of choice difficulty as well as the degree of portability between economics and psychophysics experiments