COLUR: Confidence-Oriented Learning, Unlearning and Relearning with Noisy-Label Data for Model Restoration and Refinement

TL;DR

This paper introduces COLUR, a framework that improves deep learning model performance on noisy datasets by unlearning incorrect information and relearning with refined confidence, inspired by neuroscience.

Contribution

The paper proposes a novel Confidence-Oriented Learning, Unlearning and Relearning framework (COLUR) that effectively restores and refines models trained on noisy labels using co-training.

Findings

COLUR outperforms state-of-the-art methods on four real datasets.

The framework effectively unlearns noise influence and refines confidence for better accuracy.

Extensive experiments validate the robustness of COLUR in noisy label scenarios.

Abstract

Large deep learning models have achieved significant success in various tasks. However, the performance of a model can significantly degrade if it is needed to train on datasets with noisy labels with misleading or ambiguous information. To date, there are limited investigations on how to restore performance when model degradation has been incurred by noisy label data. Inspired by the ``forgetting mechanism'' in neuroscience, which enables accelerating the relearning of correct knowledge by unlearning the wrong knowledge, we propose a robust model restoration and refinement (MRR) framework COLUR, namely Confidence-Oriented Learning, Unlearning and Relearning. Specifically, we implement COLUR with an efficient co-training architecture to unlearn the influence of label noise, and then refine model confidence on each label for relearning. Extensive experiments are conducted on four real datasets and all evaluation results show that COLUR consistently outperforms other SOTA methods after MRR.