Variation-Bounded Loss for Noise-Tolerant Learning

TL;DR

This paper introduces the Variation Ratio as a new property for robust loss functions and proposes Variation-Bounded Loss (VBL), which improves noise tolerance in supervised learning by bounding the variation ratio.

Contribution

The paper defines the variation ratio, analyzes its impact on robustness, and develops the VBL family of loss functions that enhance noise tolerance in supervised learning.

Findings

VBL improves robustness against noisy labels.

Theoretical analysis links smaller variation ratio to better robustness.

Experimental results demonstrate VBL's effectiveness across datasets.

Abstract

Mitigating the negative impact of noisy labels has been aperennial issue in supervised learning. Robust loss functions have emerged as a prevalent solution to this problem. In this work, we introduce the Variation Ratio as a novel property related to the robustness of loss functions, and propose a new family of robust loss functions, termed Variation-Bounded Loss (VBL), which is characterized by a bounded variation ratio. We provide theoretical analyses of the variation ratio, proving that a smaller variation ratio would lead to better robustness. Furthermore, we reveal that the variation ratio provides a feasible method to relax the symmetric condition and offers a more concise path to achieve the asymmetric condition. Based on the variation ratio, we reformulate several commonly used loss functions into a variation-bounded form for practical applications. Positive experiments on various datasets exhibit the effectiveness and flexibility of our approach.