NCSAM Noise-Compensated Sharpness-Aware Minimization for Noisy Label Learning

TL;DR

This paper introduces NCSAM, a novel method that uses noise-compensated sharpness-aware minimization to improve learning from noisy labels, supported by theoretical analysis and extensive experiments showing superior performance.

Contribution

It provides a theoretical link between loss landscape flatness and label noise, and proposes NCSAM to enhance robustness and generalization in noisy label learning.

Findings

NCSAM outperforms existing methods on benchmark datasets.

Theoretical analysis links loss flatness with noise robustness.

Experimental results confirm improved generalization with NCSAM.

Abstract

Learning from Noisy Labels (LNL) presents a fundamental challenge in deep learning, as real-world datasets often contain erroneous or corrupted annotations, \textit{e.g.}, data crawled from Web. Current research focuses on sophisticated label correction mechanisms. In contrast, this paper adopts a novel perspective by establishing a theoretical analysis the relationship between flatness of the loss landscape and the presence of label noise. In this paper, we theoretically demonstrate that carefully simulated label noise synergistically enhances both the generalization performance and robustness of label noises. Consequently, we propose Noise-Compensated Sharpness-aware Minimization (NCSAM) to leverage the perturbation of Sharpness-Aware Minimization (SAM) to remedy the damage of label noises. Our analysis reveals that the testing accuracy exhibits a similar behavior that has been observed on the noise-clear dataset. Extensive experimental results on multiple benchmark datasets demonstrate the consistent superiority of the proposed method over existing state-of-the-art approaches on diverse tasks.