Agnostic learning in (almost) optimal time via Gaussian surface area

TL;DR

This paper improves the bounds on the complexity of agnostically learning concept classes with bounded Gaussian surface area, achieving near-optimal time complexity by refining polynomial approximation techniques.

Contribution

It refines the analysis of Gaussian surface area-based learning, reducing the polynomial degree needed for approximation from $O(rac{ ext{surface area}^2}{ ext{error}^4})$ to $ ilde O(rac{ ext{surface area}^2}{ ext{error}^2})$, leading to near-optimal bounds.

Findings

Achieved near-optimal bounds for agnostic learning of polynomial threshold functions.

Improved polynomial degree bounds for $L_1$-approximation in Gaussian space.

Extended analysis to match lower bounds in the statistical query model.

Abstract

The complexity of learning a concept class under Gaussian marginals in the difficult agnostic model is closely related to its $L_1$-approximability by low-degree polynomials. For any concept class with Gaussian surface area at most $\Gamma$, Klivans et al. (2008) show that degree $d = O(\Gamma^2 / \varepsilon^4)$ suffices to achieve an $\varepsilon$-approximation. This leads to the best-known bounds on the complexity of learning a variety of concept classes. In this note, we improve their analysis by showing that degree $d = \tilde O (\Gamma^2 / \varepsilon^2)$ is enough. In light of lower bounds due to Diakonikolas et al. (2021), this yields (near) optimal bounds on the complexity of agnostically learning polynomial threshold functions in the statistical query model. Our proof relies on a direct analogue of a construction of Feldman et al. (2020), who considered $L_1$-approximation on the Boolean hypercube.