Introducing Fractional Classification Loss for Robust Learning with Noisy Labels

TL;DR

This paper introduces Fractional Classification Loss (FCL), a novel adaptive loss function that automatically balances robustness to noisy labels and convergence speed in deep neural network training.

Contribution

The paper proposes FCL, a new loss function that dynamically adjusts its robustness to label noise by learning the fractional derivative order during training.

Findings

FCL achieves state-of-the-art results on benchmark datasets.

FCL automatically calibrates robustness without manual hyperparameter tuning.

FCL effectively balances robustness and convergence speed.

Abstract

Robust loss functions are crucial for training deep neural networks in the presence of label noise, yet existing approaches require extensive, dataset-specific hyperparameter tuning. In this work, we introduce Fractional Classification Loss (FCL), an adaptive robust loss that automatically calibrates its robustness to label noise during training. Built within the active-passive loss framework, FCL employs the fractional derivative of the Cross-Entropy (CE) loss as its active component and the Mean Absolute Error (MAE) as its passive loss component. With this formulation, we demonstrate that the fractional derivative order $\mu$ spans a family of loss functions that interpolate between MAE-like robustness and CE-like fast convergence. Furthermore, we integrate $\mu$ into the gradient-based optimization as a learnable parameter and automatically adjust it to optimize the trade-off between robustness and convergence speed. We reveal that FCL's unique property establishes a critical trade-off that enables the stable learning of $\mu$: lower log penalties on difficult or mislabeled examples improve robustness but impose higher penalties on easy or clean data, reducing model confidence in them. Consequently, FCL can dynamically reshape its loss landscape to achieve effective classification performance under label noise. Extensive experiments on benchmark datasets show that FCL achieves state-of-the-art results without the need for manual hyperparameter tuning.