How to Select Which Active Learning Strategy is Best Suited for Your Specific Problem and Budget

TL;DR

This paper proposes a derivative-based method to dynamically select the most suitable active learning strategy based on problem characteristics and budget, improving efficiency across various tasks.

Contribution

It introduces a practical, dynamic strategy selection method for active learning, grounded in theoretical analysis and validated through empirical experiments.

Findings

Effective strategy selection across diverse budgets

Improved performance in computer vision tasks

Demonstrated adaptability to different problem settings

Abstract

In the domain of Active Learning (AL), a learner actively selects which unlabeled examples to seek labels from an oracle, while operating within predefined budget constraints. Importantly, it has been recently shown that distinct query strategies are better suited for different conditions and budgetary constraints. In practice, the determination of the most appropriate AL strategy for a given situation remains an open problem. To tackle this challenge, we propose a practical derivative-based method that dynamically identifies the best strategy for a given budget. Intuitive motivation for our approach is provided by the theoretical analysis of a simplified scenario. We then introduce a method to dynamically select an AL strategy, which takes into account the unique characteristics of the problem and the available budget. Empirical results showcase the effectiveness of our approach across diverse budgets and computer vision tasks.