Batch Active Learning from the Perspective of Sparse Approximation

TL;DR

This paper introduces a novel batch active learning framework based on sparse approximation, aiming to select informative data subsets that efficiently approximate full data training loss, applicable to various neural network models.

Contribution

It formulates batch active learning as a sparsity-constrained optimization problem balancing uncertainty and representation, solved efficiently with greedy or proximal algorithms.

Findings

Achieves competitive performance across different settings

Reduces computational complexity compared to existing methods

Applicable to both Bayesian and non-Bayesian neural networks

Abstract

Active learning enables efficient model training by leveraging interactions between machine learning agents and human annotators. We study and propose a novel framework that formulates batch active learning from the sparse approximation's perspective. Our active learning method aims to find an informative subset from the unlabeled data pool such that the corresponding training loss function approximates its full data pool counterpart. We realize the framework as sparsity-constrained discontinuous optimization problems, which explicitly balance uncertainty and representation for large-scale applications and could be solved by greedy or proximal iterative hard thresholding algorithms. The proposed method can adapt to various settings, including both Bayesian and non-Bayesian neural networks. Numerical experiments show that our work achieves competitive performance across different settings with lower computational complexity.