Bridging Generative and Discriminative Noisy-Label Learning via Direction-Agnostic EM Formulation

TL;DR

This paper introduces a direction-agnostic EM framework for noisy-label learning that combines generative modeling benefits with discriminative efficiency, achieving state-of-the-art results without explicit image synthesis.

Contribution

It proposes a novel single-stage EM approach that is direction-agnostic, replaces intractable generative components with discriminative proxies, and introduces Partial-Label Supervision for improved label uncertainty handling.

Findings

Achieves state-of-the-art accuracy on noisy-label benchmarks.

Reduces training compute compared to existing methods.

Improves label transition matrix estimation accuracy.

Abstract

Although noisy-label learning is often approached with discriminative methods for simplicity and speed, generative modeling offers a principled alternative by capturing the joint mechanism that produces features, clean labels, and corrupted observations. However, prior work typically (i) introduces extra latent variables and heavy image generators that bias training toward reconstruction, (ii) fixes a single data-generating direction (\(Y\rightarrow\!X\) or \(X\rightarrow\!Y\)), limiting adaptability, and (iii) assumes a uniform prior over clean labels, ignoring instance-level uncertainty. We propose a single-stage, EM-style framework for generative noisy-label learning that is \emph{direction-agnostic} and avoids explicit image synthesis. First, we derive a single Expectation-Maximization (EM) objective whose E-step specializes to either causal orientation without changing the overall optimization. Second, we replace the intractable \(p(X\mid Y)\) with a dataset-normalized discriminative proxy computed using a discriminative classifier on the finite training set, retaining the structural benefits of generative modeling at much lower cost. Third, we introduce \emph{Partial-Label Supervision} (PLS), an instance-specific prior over clean labels that balances coverage and uncertainty, improving data-dependent regularization. Across standard vision and natural language processing (NLP) noisy-label benchmarks, our method achieves state-of-the-art accuracy, lower transition-matrix estimation error, and substantially less training compute than current generative and discriminative baselines. Code: https://github.com/lfb-1/GNL