PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions

TL;DR

PowerEvaluationBALD introduces a scalable, evaluation-oriented active learning method using stochastic acquisition functions, outperforming traditional BatchBALD in efficiency and effectiveness for deep Bayesian models.

Contribution

The paper proposes PowerEvaluationBALD, a stochastic acquisition function that enhances evaluation-oriented deep Bayesian active learning with improved scalability and comparable or better performance.

Findings

PowerEvaluationBALD matches BatchEvaluationBALD in effectiveness.

It outperforms BatchBALD on Repeated MNIST.

It significantly reduces computational costs.

Abstract

We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian active learning, as an expansion of BatchBALD that takes into account an evaluation set of unlabeled data, for example, the pool set. We also develop a variant for the non-Bayesian setting, which we call Evaluation Information Gain. To reduce computational requirements and allow these methods to scale to larger acquisition batch sizes, we introduce stochastic acquisition functions that use importance sampling of tempered acquisition scores. We call this method PowerEvaluationBALD. We show in a few initial experiments that PowerEvaluationBALD works on par with BatchEvaluationBALD, which outperforms BatchBALD on Repeated MNIST (MNISTx2), while massively reducing the computational requirements compared to BatchBALD or BatchEvaluationBALD.