Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels

TL;DR

This paper introduces a selective noise correction method that identifies and corrects noisy labels during training, improving deep learning robustness on various datasets by balancing noise correction and data preservation.

Contribution

The proposed method selectively identifies noisy samples based on loss distribution and applies a noise transition matrix for correction, enhancing learning with noisy labels.

Findings

Significant accuracy improvements on MNIST, CIFAR-10, and CIFAR-100 datasets.

Robustness enhancement in biological scRNA-seq cell-type annotation.

Effective noise correction without discarding valuable data.

Abstract

Falsely annotated samples, also known as noisy labels, can significantly harm the performance of deep learning models. Two main approaches for learning with noisy labels are global noise estimation and data filtering. Global noise estimation approximates the noise across the entire dataset using a noise transition matrix, but it can unnecessarily adjust correct labels, leaving room for local improvements. Data filtering, on the other hand, discards potentially noisy samples but risks losing valuable data. Our method identifies potentially noisy samples based on their loss distribution. We then apply a selection process to separate noisy and clean samples and learn a noise transition matrix to correct the loss for noisy samples while leaving the clean data unaffected, thereby improving the training process. Our approach ensures robust learning and enhanced model performance by preserving valuable information from noisy samples and refining the correction process. We applied our method to standard image datasets (MNIST, CIFAR-10, and CIFAR-100) and a biological scRNA-seq cell-type annotation dataset. We observed a significant improvement in model accuracy and robustness compared to traditional methods.