Mitigating the Impact of Labeling Errors on Training via Rockafellian Relaxation

TL;DR

This paper introduces the Rockafellian Relaxation Method (RRM), a loss reweighting technique that improves neural network robustness against labeling errors and adversarial noise across various data domains.

Contribution

The paper presents a novel, architecture-independent loss reweighting approach, RRM, to mitigate the impact of labeling errors on neural network training.

Findings

RRM enhances neural network robustness in computer vision and NLP tasks.

RRM effectively mitigates dataset contamination from labeling errors and adversarial perturbations.

Experimental results show improved performance across diverse data domains.

Abstract

Labeling errors in datasets are common, arising in a variety of contexts, such as human labeling, noisy labeling, and weak labeling (i.e., image classification). Although neural networks (NNs) can tolerate modest amounts of these errors, their performance degrades substantially once error levels exceed a certain threshold. We propose a new loss reweighting, architecture-independent methodology, Rockafellian Relaxation Method (RRM) for neural network training. Experiments indicate RRM can enhance neural network methods to achieve robust performance across classification tasks in computer vision and natural language processing (sentiment analysis). We find that RRM can mitigate the effects of dataset contamination stemming from both (heavy) labeling error and/or adversarial perturbation, demonstrating effectiveness across a variety of data domains and machine learning tasks.