Exponential Savings in Agnostic Active Learning through Abstention

TL;DR

This paper demonstrates that allowing abstention in agnostic active learning can lead to exponential reductions in label requests, under certain conditions, even without distributional assumptions.

Contribution

It introduces a framework where abstention enables exponential savings in label complexity for agnostic active learning, extending to misspecified models.

Findings

Exponential label savings are achievable with abstention in agnostic active learning.

A necessary and sufficient condition for exponential savings under model misspecification.

Abstention cost just below half the random guess loss is critical for savings.

Abstract

We show that in pool-based active classification without assumptions on the underlying distribution, if the learner is given the power to abstain from some predictions by paying the price marginally smaller than the average loss $1/2$ of a random guess, exponential savings in the number of label requests are possible whenever they are possible in the corresponding realizable problem. We extend this result to provide a necessary and sufficient condition for exponential savings in pool-based active classification under the model misspecification.