DivideMix: Learning with Noisy Labels as Semi-supervised Learning

TL;DR

DivideMix introduces a semi-supervised learning framework that effectively handles noisy labels by dynamically dividing data into clean and noisy sets, improving training accuracy on benchmark datasets.

Contribution

The paper presents a novel mixture model-based data division method and a divergence training scheme to enhance learning with noisy labels using semi-supervised techniques.

Findings

Significant performance improvements over state-of-the-art methods.

Effective noise handling through dynamic data division.

Enhanced semi-supervised training with label co-refinement.

Abstract

Deep neural networks are known to be annotation-hungry. Numerous efforts have been devoted to reducing the annotation cost when learning with deep networks. Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data. In this work, we propose DivideMix, a novel framework for learning with noisy labels by leveraging semi-supervised learning techniques. In particular, DivideMix models the per-sample loss distribution with a mixture model to dynamically divide the training data into a labeled set with clean samples and an unlabeled set with noisy samples, and trains the model on both the labeled and unlabeled data in a semi-supervised manner. To avoid confirmation bias, we simultaneously train two diverged networks where each network uses the dataset division from the other network. During the semi-supervised training phase, we improve the MixMatch strategy by performing label co-refinement and label co-guessing on labeled and unlabeled samples, respectively. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods. Code is available at https://github.com/LiJunnan1992/DivideMix .