Minimax Analysis of Active Learning

TL;DR

This paper provides distribution-free bounds on the minimax label complexity of active learning, revealing new insights into how noise models and complexity measures influence label efficiency.

Contribution

It establishes new upper and lower bounds on active learning label complexity under various noise models, introducing the star number as a key complexity measure.

Findings

Active learning with VC classes has asymptotically smaller label complexity than passive learning under Tsybakov noise.

In high-noise regimes, all VC class problems share similar minimax label complexity.

The star number characterizes label complexity in low-noise regimes and aligns with many existing complexity measures.

Abstract

This work establishes distribution-free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes, under various noise models. The results reveal a number of surprising facts. In particular, under the noise model of Tsybakov (2004), the minimax label complexity of active learning with a VC class is always asymptotically smaller than that of passive learning, and is typically significantly smaller than the best previously-published upper bounds in the active learning literature. In high-noise regimes, it turns out that all active learning problems of a given VC dimension have roughly the same minimax label complexity, which contrasts with well-known results for bounded noise. In low-noise regimes, we find that the label complexity is well-characterized by a simple combinatorial complexity measure we call the star number. Interestingly, we find that almost all of the complexity measures previously explored in the active learning literature have worst-case values exactly equal to the star number. We also propose new active learning strategies that nearly achieve these minimax label complexities.