Toward Optimal Probabilistic Active Learning Using a Bayesian Approach

TL;DR

This paper introduces a Bayesian decision-theoretic active learning strategy that optimizes misclassification error reduction and effectively manages uncertainty, demonstrating superior performance across diverse datasets.

Contribution

It presents a novel Bayesian approach for active learning that directly optimizes error reduction and explains limitations of existing methods.

Findings

Outperforms state-of-the-art active learning methods in various datasets

Effectively manages uncertainty with conjugate priors

Validates approach through extensive experiments

Abstract

Gathering labeled data to train well-performing machine learning models is one of the critical challenges in many applications. Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. In this article, we propose a decision-theoretic selection strategy that (1) directly optimizes the gain in misclassification error, and (2) uses a Bayesian approach by introducing a conjugate prior distribution to determine the class posterior to deal with uncertainties. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art and why this leads to the superior performance of our approach. Extensive experiments on a large variety of datasets and different kernels validate our claims.