Joint Asymmetric Loss for Learning with Noisy Labels

TL;DR

This paper introduces the Joint Asymmetric Loss (JAL), a new robust loss framework that extends asymmetric loss functions within an active passive loss setting, effectively mitigating label noise in deep neural networks.

Contribution

The paper proposes the Joint Asymmetric Loss (JAL), combining asymmetric loss functions with active passive loss to improve robustness against noisy labels in deep learning.

Findings

JAL outperforms existing methods in noisy label scenarios.

AMSE, a novel asymmetric loss, satisfies key theoretical conditions.

Extensive experiments validate the effectiveness of JAL.

Abstract

Learning with noisy labels is a crucial task for training accurate deep neural networks. To mitigate label noise, prior studies have proposed various robust loss functions, particularly symmetric losses. Nevertheless, symmetric losses usually suffer from the underfitting issue due to the overly strict constraint. To address this problem, the Active Passive Loss (APL) jointly optimizes an active and a passive loss to mutually enhance the overall fitting ability. Within APL, symmetric losses have been successfully extended, yielding advanced robust loss functions. Despite these advancements, emerging theoretical analyses indicate that asymmetric losses, a new class of robust loss functions, possess superior properties compared to symmetric losses. However, existing asymmetric losses are not compatible with advanced optimization frameworks such as APL, limiting their potential and applicability. Motivated by this theoretical gap and the prospect of asymmetric losses, we extend the asymmetric loss to the more complex passive loss scenario and propose the Asymetric Mean Square Error (AMSE), a novel asymmetric loss. We rigorously establish the necessary and sufficient condition under which AMSE satisfies the asymmetric condition. By substituting the traditional symmetric passive loss in APL with our proposed AMSE, we introduce a novel robust loss framework termed Joint Asymmetric Loss (JAL). Extensive experiments demonstrate the effectiveness of our method in mitigating label noise. Code available at: https://github.com/cswjl/joint-asymmetric-loss