Tight Bounds for Learning Polyhedra with a Margin

TL;DR

This paper presents a new algorithm for PAC learning intersections of halfspaces with a margin, achieving improved bounds and extending to more general distribution settings.

Contribution

The authors develop an algorithm with near-optimal bounds for learning polyhedra with a margin, improving previous exponential dependencies and applicable to broader distributions.

Findings

Algorithm runs in quasi-polynomial time with respect to key parameters.

Improves upon prior exponential dependence on parameters.

Matches cryptographic and statistical lower bounds up to logarithmic factors.

Abstract

We give an algorithm for PAC learning intersections of $k$ halfspaces with a $\rho$ margin to within error $\varepsilon$ that runs in time $\textsf{poly}(k, \varepsilon^{-1}, \rho^{-1}) \cdot \exp \left(O(\sqrt{n \log(1/\rho) \log k})\right)$. Notably, this improves on prior work which had an exponential dependence on either $k$ or $\rho^{-1}$ and matches known cryptographic and Statistical Query lower bounds up to the logarithmic factors in $k$ and $\rho$ in the exponent. Our learning algorithm extends to the more general setting when we are only promised that most points have distance at least $\rho$ from the boundary of the polyhedron, making it applicable to continuous distributions as well.