Efficient Bilevel Optimization for Meta Label Correction in Noisy Label Learning

TL;DR

This paper introduces EBOMLC, an efficient meta label correction method for noisy label learning that reduces training time and improves accuracy, especially under high noise conditions.

Contribution

The paper proposes EBOMLC, a novel meta label correction approach with key improvements like one-step updates and dynamic barrier, enhancing efficiency and robustness.

Findings

EBOMLC outperforms baselines on CIFAR-10 and CIFAR-100.

It significantly reduces training time for meta label correction.

EBOMLC performs well under high noise rates.

Abstract

Training a deep neural network with noisy labels could reduce data annotation cost but may introduce noise into the learned model. In meta label correction approaches, an additional meta model besides the main model is trained with a small, clean dataset to correct the large, noisy dataset. However, the update of the meta model requires the computation of hypergradients at the inner step of the main model which signif- icantly increases the computational cost. To improve the training efficiency, we first introduce the dynamic barrier gradient descent into standard meta label correction. While this naive extenstion is able to speed up the training process to approximately first- order complexity, it lacks mechanisms to prevent the leakage of noisy signals to the main model and to stabilize the learning of the meta model. Based on this observation, we propose the EBOMLC method, which is designed with three key improvements including one-step inner loop update, mixture upper loss and alignment- aware dynamic barrier. Empirical results on CIFAR-10 and CIFAR-100 demonstrate that EBOMLC consistently outperforms other baselines, especially under high noise rate settings, while reducing training time of the meta label correction approach.