Moment-Matching Polynomials
Adam Klivans, Raghu Meka

TL;DR
This paper introduces a new moment-matching polynomial framework for Boolean functions, enabling low-degree polynomial approximations under broad distributions, and provides the first polynomial-time algorithms for agnostic learning of certain halfspace functions.
Contribution
It presents a novel moment-matching approach that extends polynomial approximation techniques beyond product distributions, leading to new efficient algorithms for learning halfspaces under general distributions.
Findings
First polynomial-time algorithm for agnostically learning functions of constant halfspaces under log-concave distributions
Framework applies to distributions with sub-exponential tails in smoothed analysis setting
Supports the effectiveness of kernel methods like SVMs in practical machine learning scenarios
Abstract
We give a new framework for proving the existence of low-degree, polynomial approximators for Boolean functions with respect to broad classes of non-product distributions. Our proofs use techniques related to the classical moment problem and deviate significantly from known Fourier-based methods, which require the underlying distribution to have some product structure. Our main application is the first polynomial-time algorithm for agnostically learning any function of a constant number of halfspaces with respect to any log-concave distribution (for any constant accuracy parameter). This result was not known even for the case of learning the intersection of two halfspaces without noise. Additionally, we show that in the "smoothed-analysis" setting, the above results hold with respect to distributions that have sub-exponential tails, a property satisfied by many natural and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Moment-Matching Polynomials· youtube
Taxonomy
TopicsMachine Learning and Algorithms · Machine Learning and Data Classification · Adversarial Robustness in Machine Learning
