Debiased Orthogonal Boundary-Driven Efficient Noise Mitigation

TL;DR

This paper introduces OSA, a model-agnostic, efficient method for mitigating noisy labels by leveraging high-dimensional orthogonality to separate clean and noisy data, improving robustness and transferability.

Contribution

The paper proposes a novel one-step, boundary-driven noise mitigation method called OSA that is model-agnostic and computationally efficient, addressing limitations of prior approaches.

Findings

OSA improves training robustness across benchmarks.

OSA enhances task transferability.

OSA reduces computational overhead.

Abstract

Mitigating the detrimental effects of noisy labels on the training process has become increasingly critical, as obtaining entirely clean or human-annotated samples for large-scale pre-training tasks is often impractical. Nonetheless, existing noise mitigation methods often encounter limitations in practical applications due to their task-specific design, model dependency, and significant computational overhead. In this work, we exploit the properties of high-dimensional orthogonality to identify a robust and effective boundary in cone space for separating clean and noisy samples. Building on this, we propose One-Step Anti-noise (OSA), a model-agnostic noisy label mitigation paradigm that employs an estimator model and a scoring function to assess the noise level of input pairs through just one-step inference. We empirically validate the superiority of OSA, demonstrating its enhanced training robustness, improved task transferability, streamlined deployment, and reduced computational overhead across diverse benchmarks, models, and tasks. Our code is released at https://github.com/leolee99/OSA.