How General Are Measures of Choice Consistency? Evidence from Experimental and Scanner Data

TL;DR

This paper examines whether measures of choice consistency from lab experiments and real-world data are comparable, finding limited correlation and suggesting that consistency may be a multidimensional construct rather than a single trait.

Contribution

It provides empirical evidence on the limited generalizability of choice consistency measures across different decision-making contexts.

Findings

Lack of correlation between experimental and supermarket consistency scores.

Moderate correlations within supermarket purchasing categories over time.

Distinct predictive validity of consistency measures in different settings.

Abstract

Choice consistency with utility maximization, as a key assumption in economics, has been extensively used to evaluate decision quality of individuals and to predict real-world outcomes across different contexts. Here we investigate the generalizability of consistency measures derived from budgetary decisions in the lab-in-the-field experiment and purchasing decisions using supermarket scanner data. In the first study, we observe a lack of correlation between consistency scores derived from risky decisions in the experiment and those from supermarket food purchasing decisions. In the second study, we observe moderate correlations between experimental tasks and low to moderate correlations across purchasing categories and over time periods within the supermarket. Moreover, consistency in the two settings exhibits distinct predictive validity in predicting consumer behavior. These results suggest that choice consistency, as a measure of decision quality, may be better characterized as a multidimensional skill set rather than a single-dimensional ability.