Searching for Robustness: Loss Learning for Noisy Classification Tasks

TL;DR

This paper introduces a novel method that automatically learns robust loss functions for noisy classification tasks using a flexible polynomial family and evolutionary strategies, improving noise resilience across diverse datasets.

Contribution

It proposes a new approach to automatically learn noise-robust loss functions via Taylor polynomial parameterization and evolutionary search, enabling plug-and-play robustness without specialized training.

Findings

Outperforms previous methods on noisy datasets

Provides a fast, reusable loss function module

Effective across synthetic and real label noise

Abstract

We present a "learning to learn" approach for automatically constructing white-box classification loss functions that are robust to label noise in the training data. We parameterize a flexible family of loss functions using Taylor polynomials, and apply evolutionary strategies to search for noise-robust losses in this space. To learn re-usable loss functions that can apply to new tasks, our fitness function scores their performance in aggregate across a range of training dataset and architecture combinations. The resulting white-box loss provides a simple and fast "plug-and-play" module that enables effective noise-robust learning in diverse downstream tasks, without requiring a special training procedure or network architecture. The efficacy of our method is demonstrated on a variety of datasets with both synthetic and real label noise, where we compare favourably to previous work.