Improved Algorithms for Efficient Active Learning Halfspaces with Massart and Tsybakov noise

TL;DR

This paper introduces a computationally efficient active learning algorithm for halfspaces that tolerates Massart and Tsybakov noise, achieving near-optimal label complexity under various data distributions.

Contribution

The paper presents the first efficient active learning algorithms for halfspaces under Massart and Tsybakov noise with provably improved label complexity bounds.

Findings

Achieves near-optimal label complexity in Massart noise setting.

Provides lower label complexity guarantees than passive learning under Tsybakov noise.

Works under a broad class of structured data distributions.

Abstract

We give a computationally-efficient PAC active learning algorithm for $d$-dimensional homogeneous halfspaces that can tolerate Massart noise (Massart and N\'ed\'elec, 2006) and Tsybakov noise (Tsybakov, 2004). Specialized to the $\eta$-Massart noise setting, our algorithm achieves an information-theoretically near-optimal label complexity of $\tilde{O}\left( \frac{d}{(1-2\eta)^2} \mathrm{polylog}(\frac1\epsilon) \right)$ under a wide range of unlabeled data distributions (specifically, the family of "structured distributions" defined in Diakonikolas et al. (2020)). Under the more challenging Tsybakov noise condition, we identify two subfamilies of noise conditions, under which our efficient algorithm provides label complexity guarantees strictly lower than passive learning algorithms.