Efficient and Parsimonious Agnostic Active Learning

TL;DR

This paper introduces a new active learning algorithm that is versatile, noise-tolerant, efficiently implementable, and more aggressive than previous methods, with theoretical guarantees and comprehensive experimental evaluation.

Contribution

It presents a novel active learning algorithm for streaming data that works universally, is computationally efficient, and outperforms existing approaches in aggressiveness.

Findings

Algorithm works for any classifier and noisy data

Efficient implementation with ERM oracle

First comprehensive experimental analysis

Abstract

We develop a new active learning algorithm for the streaming setting satisfying three important properties: 1) It provably works for any classifier representation and classification problem including those with severe noise. 2) It is efficiently implementable with an ERM oracle. 3) It is more aggressive than all previous approaches satisfying 1 and 2. To do this we create an algorithm based on a newly defined optimization problem and analyze it. We also conduct the first experimental analysis of all efficient agnostic active learning algorithms, evaluating their strengths and weaknesses in different settings.