Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensembles

TL;DR

This paper introduces DEBAL, an active learning strategy using ensembles to address mode collapse in Bayesian methods, leading to faster CNN training convergence on image datasets.

Contribution

DEBAL enhances deep Bayesian active learning by utilizing ensembles to better capture data uncertainty and overcome mode collapse issues.

Findings

Faster convergence of CNNs on MNIST and CIFAR-10 datasets.

Improved data uncertainty estimation over existing methods.

Addresses mode collapse in Bayesian active learning.

Abstract

In image classification tasks, the ability of deep CNNs to deal with complex image data has proven to be unrivalled. However, they require large amounts of labeled training data to reach their full potential. In specialised domains such as healthcare, labeled data can be difficult and expensive to obtain. Active Learning aims to alleviate this problem, by reducing the amount of labelled data needed for a specific task while delivering satisfactory performance. We propose DEBAL, a new active learning strategy designed for deep neural networks. This method improves upon the current state-of-the-art deep Bayesian active learning method, which suffers from the mode collapse problem. We correct for this deficiency by making use of the expressive power and statistical properties of model ensembles. Our proposed method manages to capture superior data uncertainty, which translates into improved classification performance. We demonstrate empirically that our ensemble method yields faster convergence of CNNs trained on the MNIST and CIFAR-10 datasets.