Learning how to Active Learn: A Deep Reinforcement Learning Approach

TL;DR

This paper proposes a deep reinforcement learning approach to active learning, enabling the automatic learning of data selection policies that can transfer across languages, improving performance in cross-lingual named entity recognition.

Contribution

The paper introduces a novel reinforcement learning formulation for active learning, allowing learned policies to transfer across languages and outperform traditional heuristics.

Findings

Uniform improvements over traditional active learning methods

Effective transfer of learned policies across different languages

Enhanced performance in cross-lingual named entity recognition

Abstract

Active learning aims to select a small subset of data for annotation such that a classifier learned on the data is highly accurate. This is usually done using heuristic selection methods, however the effectiveness of such methods is limited and moreover, the performance of heuristics varies between datasets. To address these shortcomings, we introduce a novel formulation by reframing the active learning as a reinforcement learning problem and explicitly learning a data selection policy, where the policy takes the role of the active learning heuristic. Importantly, our method allows the selection policy learned using simulation on one language to be transferred to other languages. We demonstrate our method using cross-lingual named entity recognition, observing uniform improvements over traditional active learning.