Learning with Open-world Noisy Data via Class-independent Margin in Dual Representation Space

TL;DR

This paper introduces a dual-space joint learning approach with class-independent margin criteria to robustly handle open-world noisy labels, improving model generalization in real-world noisy data scenarios.

Contribution

It proposes a novel dual representation space and class-independent margin criteria to effectively detect and mitigate open-set noise in noisy label learning.

Findings

Outperforms state-of-the-art methods on CIFAR80N with 4.55% accuracy gain.

Achieves 6.17% higher AUROC in noise detection.

Demonstrates robustness against open-set and closed-set noisy labels.

Abstract

Learning with Noisy Labels (LNL) aims to improve the model generalization when facing data with noisy labels, and existing methods generally assume that noisy labels come from known classes, called closed-set noise. However, in real-world scenarios, noisy labels from similar unknown classes, i.e., open-set noise, may occur during the training and inference stage. Such open-world noisy labels may significantly impact the performance of LNL methods. In this study, we propose a novel dual-space joint learning method to robustly handle the open-world noise. To mitigate model overfitting on closed-set and open-set noises, a dual representation space is constructed by two networks. One is a projection network that learns shared representations in the prototype space, while the other is a One-Vs-All (OVA) network that makes predictions using unique semantic representations in the class-independent space. Then, bi-level contrastive learning and consistency regularization are introduced in two spaces to enhance the detection capability for data with unknown classes. To benefit from the memorization effects across different types of samples, class-independent margin criteria are designed for sample identification, which selects clean samples, weights closed-set noise, and filters open-set noise effectively. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods and achieves an average accuracy improvement of 4.55\% and an AUROC improvement of 6.17\% on CIFAR80N.