Scalable Batch-Mode Deep Bayesian Active Learning via Equivalence Class Annealing

TL;DR

This paper introduces Batch-BALanCe, a scalable active learning algorithm that improves batch query selection for deep Bayesian models by using a novel decision-theoretic acquisition function and equivalence class adjustments, enhancing efficiency and performance.

Contribution

It proposes a new scalable batch-mode active learning method combining decision theory, information measures, and diversity sampling with adaptive equivalence class adjustments.

Findings

Effective on multi-class classification tasks

Achieves strong performance on benchmark datasets

Handles both low- and large-batch regimes

Abstract

Active learning has demonstrated data efficiency in many fields. Existing active learning algorithms, especially in the context of batch-mode deep Bayesian active models, rely heavily on the quality of uncertainty estimations of the model, and are often challenging to scale to large batches. In this paper, we propose Batch-BALanCe, a scalable batch-mode active learning algorithm, which combines insights from decision-theoretic active learning, combinatorial information measure, and diversity sampling. At its core, Batch-BALanCe relies on a novel decision-theoretic acquisition function that facilitates differentiation among different equivalence classes. Intuitively, each equivalence class consists of hypotheses (e.g., posterior samples of deep neural networks) with similar predictions, and Batch-BALanCe adaptively adjusts the size of the equivalence classes as learning progresses. To scale up the computation of queries to large batches, we further propose an efficient batch-mode acquisition procedure, which aims to maximize a novel information measure defined through the acquisition function. We show that our algorithm can effectively handle realistic multi-class classification tasks, and achieves compelling performance on several benchmark datasets for active learning under both low- and large-batch regimes. Reference code is released at https://github.com/zhangrenyuuchicago/BALanCe.