Discovering General-Purpose Active Learning Strategies

TL;DR

This paper introduces a universal reinforcement learning-based framework for discovering active learning strategies that are transferable across domains and models, effectively reducing annotation costs.

Contribution

It formalizes the active learning process as a Markov decision process and develops a reinforcement learning approach to find optimal, domain-agnostic AL strategies.

Findings

Learned strategies outperform state-of-the-art baselines

Strategies transfer effectively across unrelated domains

Reinforcement learning optimizes annotation cost reduction

Abstract

We propose a general-purpose approach to discovering active learning (AL) strategies from data. These strategies are transferable from one domain to another and can be used in conjunction with many machine learning models. To this end, we formalize the annotation process as a Markov decision process, design universal state and action spaces and introduce a new reward function that precisely model the AL objective of minimizing the annotation cost. We seek to find an optimal (non-myopic) AL strategy using reinforcement learning. We evaluate the learned strategies on multiple unrelated domains and show that they consistently outperform state-of-the-art baselines.