Learning under noisy supervision is governed by a feedback-truth gap

TL;DR

This paper demonstrates that a feedback-truth gap, caused by differing learning rates, universally affects systems from neural networks to humans, influencing how feedback and truth are balanced during learning.

Contribution

It introduces a two-timescale model predicting the feedback-truth gap and empirically validates this across neural networks and human experiments, revealing how different systems regulate this gap.

Findings

The feedback-truth gap appears universally across systems.

Neural networks tend to memorize, amplifying the gap.

Humans transiently over-commit but actively recover from it.

Abstract

When feedback is absorbed faster than task structure can be evaluated, the learner will favor feedback over truth. A two-timescale model shows this feedback-truth gap is inevitable whenever the two rates differ and vanishes only when they match. We test this prediction across neural networks trained with noisy labels (30 datasets, 2,700 runs), human probabilistic reversal learning (N = 292), and human reward/punishment learning with concurrent EEG (N = 25). In each system, truth is defined operationally: held-out labels, the objectively correct option, or the participant's pre-feedback expectation - the only non-circular reference decodable from post-feedback EEG. The gap appeared universally but was regulated differently: dense networks accumulated it as memorization; sparse-residual scaffolding suppressed it; humans generated transient over-commitment that was actively recovered. Neural over-commitment (~0.04-0.10) was amplified tenfold into behavioral commitment (d = 3.3-3.9). The gap is a fundamental constraint on learning under noisy supervision; its consequences depend on the regulation each system employs.