Learning Constant-Depth Circuits in Malicious Noise Models

TL;DR

This paper presents a new algorithm for learning constant-depth circuits under malicious noise, extending previous work to handle adversarial corruptions with optimal efficiency and robustness.

Contribution

It introduces a novel outlier-removal approach combined with Braverman's theorem to learn $ ext{AC}^0$ circuits in the presence of malicious noise, matching the best known algorithms' efficiency.

Findings

Achieves learning of $ ext{AC}^0$ circuits under malicious noise.

Optimal dependence on noise rate in the harshest noise models.

Runs in essentially the same time as previous algorithms, which are cryptographically optimal.

Abstract

The seminal work of Linial, Mansour, and Nisan gave a quasipolynomial-time algorithm for learning constant-depth circuits ($\mathsf{AC}^0$) with respect to the uniform distribution on the hypercube. Extending their algorithm to the setting of malicious noise, where both covariates and labels can be adversarially corrupted, has remained open. Here we achieve such a result, inspired by recent work on learning with distribution shift. Our running time essentially matches their algorithm, which is known to be optimal assuming various cryptographic primitives. Our proof uses a simple outlier-removal method combined with Braverman's theorem for fooling constant-depth circuits. We attain the best possible dependence on the noise rate and succeed in the harshest possible noise model (i.e., contamination or so-called "nasty noise").