New Statistical and Computational Results for Learning Junta Distributions

TL;DR

This paper establishes the computational equivalence between learning junta distributions and learning noisy parity functions, and introduces an optimal statistical algorithm for junta learning, highlighting fundamental limits in the field.

Contribution

It proves the equivalence between learning junta distributions and noisy parity functions, and presents an optimal statistical algorithm for junta learning.

Findings

Learning junta distributions is computationally equivalent to learning noisy parity functions.

The proposed algorithm achieves near-optimal statistical complexity.

Computational complexity of the algorithm matches previous non-sample-optimal methods.

Abstract

We study the problem of learning junta distributions on $\{0, 1\}^n$, where a distribution is a $k$-junta if its probability mass function depends on a subset of at most $k$ variables. We make two main contributions: - We show that learning $k$-junta distributions is \emph{computationally} equivalent to learning $k$-parity functions with noise (LPN), a landmark problem in computational learning theory. - We design an algorithm for learning junta distributions whose statistical complexity is optimal, up to polylogarithmic factors. Computationally, our algorithm matches the complexity of previous (non-sample-optimal) algorithms. Combined, our two contributions imply that our algorithm cannot be significantly improved, statistically or computationally, barring a breakthrough for LPN.