Efficacy of Bayesian Neural Networks in Active Learning

TL;DR

This paper investigates how Bayesian neural networks can improve active learning by better modeling uncertainty, demonstrating their superior efficiency over ensemble methods in selecting informative data samples.

Contribution

The study provides a comprehensive experimental comparison showing Bayesian neural networks outperform ensemble techniques in active learning scenarios.

Findings

Bayesian neural networks more effectively capture uncertainty.

Bayesian methods outperform ensemble techniques in active learning.

Ensemble methods have notable drawbacks compared to Bayesian approaches.

Abstract

Obtaining labeled data for machine learning tasks can be prohibitively expensive. Active learning mitigates this issue by exploring the unlabeled data space and prioritizing the selection of data that can best improve the model performance. A common approach to active learning is to pick a small sample of data for which the model is most uncertain. In this paper, we explore the efficacy of Bayesian neural networks for active learning, which naturally models uncertainty by learning distribution over the weights of neural networks. By performing a comprehensive set of experiments, we show that Bayesian neural networks are more efficient than ensemble based techniques in capturing uncertainty. Our findings also reveal some key drawbacks of the ensemble techniques, which was recently shown to be more effective than Monte Carlo dropouts.