Testable Learning of General Halfspaces under Massart Noise

TL;DR

This paper introduces the first testable learning algorithm for general halfspaces with Massart noise under Gaussian distributions, achieving near-optimal error with a quasi-polynomial complexity.

Contribution

It presents a novel testable learning algorithm for Massart halfspaces with Gaussian marginals, including a new sandwiching polynomial approximation of the sign function.

Findings

First testable learning algorithm for Massart halfspaces

Algorithm complexity matches known lower bounds

Develops a new polynomial approximation technique

Abstract

We study the algorithmic task of testably learning general Massart halfspaces under the Gaussian distribution. In the testable learning setting, the aim is the design of a tester-learner pair satisfying the following properties: (1) if the tester accepts, the learner outputs a hypothesis and a certificate that it achieves near-optimal error, and (2) it is highly unlikely that the tester rejects if the data satisfies the underlying assumptions. Our main result is the first testable learning algorithm for general halfspaces with Massart noise and Gaussian marginals. The complexity of our algorithm is $d^{\mathrm{polylog}(\min\{1/\gamma, 1/\epsilon \})}$, where $\epsilon$ is the excess error and $\gamma$ is the bias of the target halfspace, which qualitatively matches the known quasi-polynomial Statistical Query lower bound for the non-testable setting. The analysis of our algorithm hinges on a novel sandwiching polynomial approximation to the sign function with multiplicative error that may be of broader interest.