MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels

TL;DR

MetaLabelNet introduces a meta-learning approach that generates soft-labels from noisy data to improve deep neural network robustness, leveraging a small amount of clean meta-data for enhanced performance.

Contribution

The paper presents a novel meta-objective-based algorithm that learns to produce soft-labels from features, improving noise robustness in deep learning models.

Findings

Outperforms existing noise-robust methods on benchmark datasets

Effective with minimal clean meta-data

Handles both synthetic and real-world label noise

Abstract

Real-world datasets commonly have noisy labels, which negatively affects the performance of deep neural networks (DNNs). In order to address this problem, we propose a label noise robust learning algorithm, in which the base classifier is trained on soft-labels that are produced according to a meta-objective. In each iteration, before conventional training, the meta-objective reshapes the loss function by changing soft-labels, so that resulting gradient updates would lead to model parameters with minimum loss on meta-data. Soft-labels are generated from extracted features of data instances, and the mapping function is learned by a single layer perceptron (SLP) network, which is called MetaLabelNet. Following, base classifier is trained by using these generated soft-labels. These iterations are repeated for each batch of training data. Our algorithm uses a small amount of clean data as meta-data, which can be obtained effortlessly for many cases. We perform extensive experiments on benchmark datasets with both synthetic and real-world noises. Results show that our approach outperforms existing baselines.