A PTAS for Agnostically Learning Halfspaces

TL;DR

This paper introduces a polynomial-time approximation scheme for agnostically learning halfspaces on the sphere, achieving near-optimal error bounds with improved efficiency over previous algorithms.

Contribution

The paper presents the first PTAS for agnostically learning halfspaces on the sphere, combining polynomial regression with a novel localization technique.

Findings

Achieves error within (1+μ) times the optimal plus ε

Runs in polynomial time in dimension and 1/ε

Improves upon previous algorithms with unspecified approximation ratios

Abstract

We present a PTAS for agnostically learning halfspaces w.r.t. the uniform distribution on the $d$ dimensional sphere. Namely, we show that for every $\mu>0$ there is an algorithm that runs in time $\mathrm{poly}(d,\frac{1}{\epsilon})$, and is guaranteed to return a classifier with error at most $(1+\mu)\mathrm{opt}+\epsilon$, where $\mathrm{opt}$ is the error of the best halfspace classifier. This improves on Awasthi, Balcan and Long [ABL14] who showed an algorithm with an (unspecified) constant approximation ratio. Our algorithm combines the classical technique of polynomial regression (e.g. [LMN89, KKMS05]), together with the new localization technique of [ABL14].