Learning to Trust: How Humans Mentally Recalibrate AI Confidence Signals

TL;DR

Humans can learn to recalibrate their trust in AI confidence signals through repeated interactions, improving their ability to judge AI correctness despite systematic miscalibrations, with some limitations in reverse confidence scenarios.

Contribution

This study demonstrates that humans adaptively recalibrate trust in AI confidence signals via experience and introduces a computational model explaining the underlying learning dynamics.

Findings

Participants improved accuracy and calibration over trials

Humans adapt trust by updating baseline and sensitivity

Difficulty persists in reverse confidence conditions

Abstract

Productive human-AI collaboration requires appropriate reliance, yet contemporary AI systems are often miscalibrated, exhibiting systematic overconfidence or underconfidence. We investigate whether humans can learn to mentally recalibrate AI confidence signals through repeated experience. In a behavioral experiment (N = 200), participants predicted the AI's correctness across four AI calibration conditions: standard, overconfidence, underconfidence, and a counterintuitive "reverse confidence" mapping. Results demonstrate robust learning across all conditions, with participants significantly improving their accuracy, discrimination, and calibration alignment over 50 trials. We present a computational model utilizing a linear-in-log-odds (LLO) transformation and a Rescorla-Wagner learning rule to explain these dynamics. The model reveals that humans adapt by updating their baseline trust and confidence sensitivity, using asymmetric learning rates to prioritize the most informative errors. While humans can compensate for monotonic miscalibration, we identify a significant boundary in the reverse confidence scenario, where a substantial proportion of participants struggled to override initial inductive biases. These findings provide a mechanistic account of how humans adapt their trust in AI confidence signals through experience.