IALE: Imitating Active Learner Ensembles

TL;DR

This paper introduces IALE, an imitation learning approach that learns to select the best active learning heuristic at each stage, improving data labeling efficiency across datasets.

Contribution

It presents a novel imitation learning scheme using DAGGER to emulate the best AL heuristic, adaptable across similar datasets.

Findings

Outperforms existing imitation learners.

Surpasses individual AL heuristics in effectiveness.

Effective across multiple datasets.

Abstract

Active learning (AL) prioritizes the labeling of the most informative data samples. However, the performance of AL heuristics depends on the structure of the underlying classifier model and the data. We propose an imitation learning scheme that imitates the selection of the best expert heuristic at each stage of the AL cycle in a batch-mode pool-based setting. We use DAGGER to train the policy on a dataset and later apply it to datasets from similar domains. With multiple AL heuristics as experts, the policy is able to reflect the choices of the best AL heuristics given the current state of the AL process. Our experiment on well-known datasets show that we both outperform state of the art imitation learners and heuristics.