Querying Easily Flip-flopped Samples for Deep Active Learning

TL;DR

This paper introduces the least disagree metric (LDM), a computationally efficient uncertainty measure for deep active learning, demonstrating state-of-the-art results across various datasets and architectures.

Contribution

The paper proposes the LDM as a novel, asymptotically consistent uncertainty measure for deep active learning, with an efficient implementation method.

Findings

LDM-based active learning achieves state-of-the-art performance.

The estimator for LDM is computationally efficient and easy to implement.

Experimental results validate the effectiveness of LDM across datasets and models.

Abstract

Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data. One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is. The sample's distance to the decision boundary is a natural measure of predictive uncertainty, but it is often intractable to compute, especially for complex decision boundaries formed in multiclass classification tasks. To address this issue, this paper proposes the {\it least disagree metric} (LDM), defined as the smallest probability of disagreement of the predicted label, and an estimator for LDM proven to be asymptotically consistent under mild assumptions. The estimator is computationally efficient and can be easily implemented for deep learning models using parameter perturbation. The LDM-based active learning is performed by querying unlabeled data with the smallest LDM. Experimental results show that our LDM-based active learning algorithm obtains state-of-the-art overall performance on all considered datasets and deep architectures.