Are Candidate Models Really Needed for Active Learning?

TL;DR

This paper explores a simplified active learning approach that eliminates the need for initial candidate models by using randomly initialized CNNs and transformers, achieving comparable results to traditional methods.

Contribution

It demonstrates that confidence-based sampling strategies, especially low confidence, can effectively replace candidate models in active learning, simplifying the process.

Findings

LC strategy often outperforms HC and HCLC in experiments.

The proposed methods are robust across various datasets and domains.

Eliminating candidate models reduces time and resource requirements.

Abstract

Deep learning has profoundly impacted domains such as computer vision and natural language processing by uncovering complex patterns in vast datasets. However, the reliance on extensive labeled data poses significant challenges, including resource constraints and annotation errors, particularly in training Convolutional Neural Networks (CNNs) and transformers due to a larger number of parameters. Active learning offers a promising solution to reduce labeling burdens by strategically selecting the most informative samples for annotation. However, the current active learning frameworks are time-intensive which select the samples iteratively with the help of initial candidate models. This study investigates the feasibility of using CNNs and transformers with randomly initialized weights, eliminating the need for initial candidate models while achieving results comparable to active learning frameworks that depend on such candidate models. We evaluate three confidence-based sampling strategies: high confidence (HC), low confidence (LC), and a combination of high confidence in the early stages of training and low confidence at later stages of training (HCLC). Among these, mostly LC demonstrated the best performance in our experiments, showcasing its effectiveness as an active learning strategy without the need for candidate models. Further, extensive experiments verify the robustness of the proposed active learning methods. By challenging traditional frameworks, the proposed work introduces a streamlined approach to active learning, advancing efficiency and flexibility across diverse datasets and domains.