Frugal Reinforcement-based Active Learning

TL;DR

This paper introduces a novel probabilistic active learning method that combines diversity, representativity, and uncertainty criteria, using reinforcement learning to adaptively select critical samples for training deep neural networks efficiently.

Contribution

It proposes a unified, probabilistic active learning framework with a reinforcement learning-based weighting mechanism to improve label efficiency in training deep models.

Findings

Outperforms baseline active learning methods on image classification datasets.

Effectively balances diversity, representativity, and uncertainty criteria.

Reduces labeled data requirements while maintaining high accuracy.

Abstract

Most of the existing learning models, particularly deep neural networks, are reliant on large datasets whose hand-labeling is expensive and time demanding. A current trend is to make the learning of these models frugal and less dependent on large collections of labeled data. Among the existing solutions, deep active learning is currently witnessing a major interest and its purpose is to train deep networks using as few labeled samples as possible. However, the success of active learning is highly dependent on how critical are these samples when training models. In this paper, we devise a novel active learning approach for label-efficient training. The proposed method is iterative and aims at minimizing a constrained objective function that mixes diversity, representativity and uncertainty criteria. The proposed approach is probabilistic and unifies all these criteria in a single objective function whose solution models the probability of relevance of samples (i.e., how critical) when learning a decision function. We also introduce a novel weighting mechanism based on reinforcement learning, which adaptively balances these criteria at each training iteration, using a particular stateless Q-learning model. Extensive experiments conducted on staple image classification data, including Object-DOTA, show the effectiveness of our proposed model w.r.t. several baselines including random, uncertainty and flat as well as other work.