Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis

TL;DR

This paper investigates why ideal noise correction methods fail even with perfect transition matrices, revealing fundamental flaws beyond estimation errors and providing insights for designing more reliable noisy label learning techniques.

Contribution

It demonstrates that noise correction failure persists despite perfect transition matrices and offers a unified analysis linking convergence, optimization, and information limits.

Findings

Ideal noise correction methods still fail under perfect conditions.

Failure is rooted in deeper flaws beyond transition matrix estimation.

Provides guidance for developing more robust noisy label learning methods.

Abstract

Statistically consistent methods based on the noise transition matrix ($T$) offer a theoretically grounded solution to Learning with Noisy Labels (LNL), with guarantees of convergence to the optimal clean-data classifier. In practice, however, these methods are often outperformed by empirical approaches such as sample selection, and this gap is usually attributed to the difficulty of accurately estimating $T$. The common assumption is that, given a perfect $T$, noise-correction methods would recover their theoretical advantage. In this work, we put this longstanding hypothesis to a decisive test. We conduct experiments under idealized conditions, providing correction methods with a perfect, oracle transition matrix. Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training. This compellingly demonstrates that the failure is not fundamentally a $T$-estimation problem, but stems from a more deeply rooted flaw. To explain this behaviour, we provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels. Together, these results give a formal account of why ideal noise correction fails and offer concrete guidance for designing more reliable methods for learning with noisy labels.