The efficiency-gain illusion: People underestimate the rate of AI use and overestimate its benefits on simple tasks

TL;DR

This study reveals that people often overuse AI for simple tasks, overestimating its benefits and underestimating their own AI usage, leading to potential overreliance and inefficiency.

Contribution

The paper uncovers systematic biases in people's perceptions of AI efficiency and usage, supported by three large pre-registered user studies.

Findings

People frequently use AI even when it offers no time or effort savings.

Individuals underestimate their own AI use but overestimate AI's efficiency gains.

Prior AI use increases subsequent AI adoption and miscalibration.

Abstract

People are increasingly turning to AI assistance for simple tasks, e.g., arithmetic, spell-check, and answering simple questions. But does AI assistance actually save users time and effort? We investigate people's propensity to use AI for cognitively simple tasks and assess whether their reliance is well-calibrated. Across three pre-registered user studies (N = 2691), we find that people frequently choose to use AI even when doing so is inefficient (i.e. provides no meaningful time or effort savings). We identify systematic miscalibration at two levels: (1) a self-estimate miscalibration where people on average believe that they are using AI less than they actually are, and (2) efficiency-gain illusions where people overestimate how much time and effort savings AI use affords. We also identify a session-level carryover effect where a participant's prior AI use leads to further AI adoption and entrenches their miscalibration about time savings. Our results shed light on the mechanisms and biases underlying people's choice of whether to use AI as well as the risk of an overreliance feedback loop.