Learning Active Learning from Data

TL;DR

This paper introduces a data-driven active learning method that trains a regressor to predict error reduction, enabling learned query strategies that outperform traditional heuristics across various domains.

Contribution

It proposes a novel approach to active learning by learning query strategies from data, moving beyond fixed heuristics to adaptive, experience-based methods.

Findings

Strategies learned from synthetic data generalize well to real data.

The method improves active learning performance across multiple domains.

Learned strategies outperform traditional heuristics in experiments.

Abstract

In this paper, we suggest a novel data-driven approach to active learning (AL). The key idea is to train a regressor that predicts the expected error reduction for a candidate sample in a particular learning state. By formulating the query selection procedure as a regression problem we are not restricted to working with existing AL heuristics; instead, we learn strategies based on experience from previous AL outcomes. We show that a strategy can be learnt either from simple synthetic 2D datasets or from a subset of domain-specific data. Our method yields strategies that work well on real data from a wide range of domains.