Black-Box Batch Active Learning for Regression

TL;DR

This paper introduces a black-box batch active learning method for regression that relies solely on model predictions, enabling its use across diverse models including non-differentiable ones, and demonstrates strong empirical performance.

Contribution

It extends white-box batch active learning methods to black-box models using Bayesian and kernel-based techniques, broadening applicability.

Findings

Effective on various regression datasets

Outperforms some white-box methods in experiments

Compatible with non-differentiable models like random forests

Abstract

Batch active learning is a popular approach for efficiently training machine learning models on large, initially unlabelled datasets by repeatedly acquiring labels for batches of data points. However, many recent batch active learning methods are white-box approaches and are often limited to differentiable parametric models: they score unlabeled points using acquisition functions based on model embeddings or first- and second-order derivatives. In this paper, we propose black-box batch active learning for regression tasks as an extension of white-box approaches. Crucially, our method only relies on model predictions. This approach is compatible with a wide range of machine learning models, including regular and Bayesian deep learning models and non-differentiable models such as random forests. It is rooted in Bayesian principles and utilizes recent kernel-based approaches. This allows us to extend a wide range of existing state-of-the-art white-box batch active learning methods (BADGE, BAIT, LCMD) to black-box models. We demonstrate the effectiveness of our approach through extensive experimental evaluations on regression datasets, achieving surprisingly strong performance compared to white-box approaches for deep learning models.