Batch Active Learning at Scale

TL;DR

This paper presents a scalable batch active learning algorithm that combines uncertainty and diversity, enabling efficient training with very large batch sizes and providing theoretical label complexity guarantees.

Contribution

It introduces a scalable sampling method for batch active learning that handles very large batch sizes and offers theoretical label complexity guarantees.

Findings

Scales to batch sizes of 100K-1M, much larger than previous methods.

Achieves significant improvements in training efficiency over recent baselines.

Provides initial theoretical analysis with label complexity guarantees.

Abstract

The ability to train complex and highly effective models often requires an abundance of training data, which can easily become a bottleneck in cost, time, and computational resources. Batch active learning, which adaptively issues batched queries to a labeling oracle, is a common approach for addressing this problem. The practical benefits of batch sampling come with the downside of less adaptivity and the risk of sampling redundant examples within a batch -- a risk that grows with the batch size. In this work, we analyze an efficient active learning algorithm, which focuses on the large batch setting. In particular, we show that our sampling method, which combines notions of uncertainty and diversity, easily scales to batch sizes (100K-1M) several orders of magnitude larger than used in previous studies and provides significant improvements in model training efficiency compared to recent baselines. Finally, we provide an initial theoretical analysis, proving label complexity guarantees for a related sampling method, which we show is approximately equivalent to our sampling method in specific settings.