Noisy Batch Active Learning with Deterministic Annealing

TL;DR

This paper proposes a robust batch active learning method that incorporates model uncertainty and denoising techniques to improve training with noisy labels across image classification benchmarks.

Contribution

It introduces a novel noisy batch active learning approach with deterministic annealing and a denoising layer for deep networks, enhancing robustness to label noise.

Findings

Improved accuracy over existing active learning strategies on benchmark datasets.

Effective incorporation of model uncertainty to handle small training data.

Significant robustness gains with the denoising layer in deep networks.

Abstract

We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. More importantly, we incorporate model uncertainty into the sampling probability to compensate for poor estimation of the importance scores when the training data is too small to build a meaningful model. Experiments on benchmark image classification datasets (MNIST, SVHN, CIFAR10, and EMNIST) show improvement over existing active learning strategies. We introduce an extra denoising layer to deep networks to make active learning robust to label noises and show significant improvements.