Labeled TrustSet Guided: Batch Active Learning with Reinforcement Learning

TL;DR

This paper introduces BRAL-T, a novel batch active learning framework combining TrustSet selection and reinforcement learning to improve data efficiency and model performance across multiple image classification tasks.

Contribution

It proposes TrustSet for selecting informative labeled data and RL-based policies for unlabeled data, achieving state-of-the-art results in active learning benchmarks.

Findings

BRAL-T outperforms existing methods on 10 image classification benchmarks.

TrustSet effectively balances class distribution and reduces redundancy.

Reinforcement learning enhances selection quality from unlabeled data.

Abstract

Batch active learning (BAL) is a crucial technique for reducing labeling costs and improving data efficiency in training large-scale deep learning models. Traditional BAL methods often rely on metrics like Mahalanobis Distance to balance uncertainty and diversity when selecting data for annotation. However, these methods predominantly focus on the distribution of unlabeled data and fail to leverage feedback from labeled data or the model's performance. To address these limitations, we introduce TrustSet, a novel approach that selects the most informative data from the labeled dataset, ensuring a balanced class distribution to mitigate the long-tail problem. Unlike CoreSet, which focuses on maintaining the overall data distribution, TrustSet optimizes the model's performance by pruning redundant data and using label information to refine the selection process. To extend the benefits of TrustSet to the unlabeled pool, we propose a reinforcement learning (RL)-based sampling policy that approximates the selection of high-quality TrustSet candidates from the unlabeled data. Combining TrustSet and RL, we introduce the Batch Reinforcement Active Learning with TrustSet (BRAL-T) framework. BRAL-T achieves state-of-the-art results across 10 image classification benchmarks and 2 active fine-tuning tasks, demonstrating its effectiveness and efficiency in various domains.