Learning $\mathsf{AC}^0$ Under Graphical Models

TL;DR

This paper presents quasipolynomial-time algorithms for learning AC^0 circuits under graphical models with strong spatial mixing, extending beyond the traditional product distribution setting.

Contribution

It introduces a novel approach to learn AC^0 circuits under correlated distributions using sampling algorithms, bypassing Fourier analysis limitations.

Findings

Algorithms work under graphical models with polynomial growth and strong spatial mixing.

Method extends to other function classes like monotone functions and halfspaces.

Achieves learning guarantees beyond product distributions.

Abstract

In a landmark result, Linial, Mansour and Nisan (J. ACM 1993) gave a quasipolynomial-time algorithm for learning constant-depth circuits given labeled i.i.d. samples under the uniform distribution. Their work has had a deep and lasting legacy in computational learning theory, in particular introducing the $\textit{low-degree algorithm}$. However, an important critique of many results and techniques in the area is the reliance on product structure, which is unlikely to hold in realistic settings. Obtaining similar learning guarantees for more natural correlated distributions has been a longstanding challenge in the field. In particular, we give quasipolynomial-time algorithms for learning $\mathsf{AC}^0$ substantially beyond the product setting, when the inputs come from any graphical model with polynomial growth that exhibits strong spatial mixing. The main technical challenge is in giving a workaround to Fourier analysis, which we do by showing how new sampling algorithms allow us to transfer statements about low-degree polynomial approximation under the uniform setting to graphical models. Our approach is general enough to extend to other well-studied function classes, like monotone functions and halfspaces.