BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning

TL;DR

BatchBALD introduces an efficient method for selecting diverse and informative data batches in deep Bayesian active learning, significantly improving data efficiency and outperforming existing approaches on standard benchmarks.

Contribution

It proposes BatchBALD, a tractable approximation for joint mutual information, enabling diverse batch selection and achieving state-of-the-art results in deep Bayesian active learning.

Findings

BatchBALD outperforms traditional batch acquisition methods.

It achieves state-of-the-art performance on standard benchmarks.

The method provides substantial data efficiency improvements.

Abstract

We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time $1 - \frac{1}{e}$-approximate algorithm amenable to dynamic programming and efficient caching. We compare BatchBALD to the commonly used approach for batch data acquisition and find that the current approach acquires similar and redundant points, sometimes performing worse than randomly acquiring data. We finish by showing that, using BatchBALD to consider dependencies within an acquisition batch, we achieve new state of the art performance on standard benchmarks, providing substantial data efficiency improvements in batch acquisition.