Learning DNF through Generalized Fourier Representations

TL;DR

This paper introduces a generalized Fourier expansion for learning DNF formulas under arbitrary distributions represented as Bayesian networks, extending existing methods and providing new learnability results.

Contribution

It develops a generalized Fourier framework for any distribution via Bayesian networks, enabling DNF learning under broader conditions.

Findings

The generalized Fourier expansion applies to any distribution represented by Bayesian networks.

Membership query algorithms can be adapted to the new expansion for learning DNF.

DNF is learnable under difference bounded tree Bayesian network distributions.

Abstract

The Fourier representation for the uniform distribution over the Boolean cube has found numerous applications in algorithms and complexity analysis. Notably, in learning theory, learnability of Disjunctive Normal Form (DNF) under uniform as well as product distributions has been established through such representations. This paper makes five main contributions. First, it introduces a generalized Fourier expansion that can be used with any distribution $D$ through the representation of the distribution as a Bayesian network (BN). Second, it shows that the main algorithmic tools for learning with the Fourier representation, that use membership queries to approximate functions by recovering their heavy Fourier coefficients, can be used with slight modifications with the generalized expansion. These results hold for any distribution. Third, it analyzes the $L_1$ spectral norm of conjunctions under the new expansion, showing that it is bounded for a class of distributions which can be represented by difference bounded tree BN, where a parent node in the BN representation can change the conditional expectation of a child node by at most $\alpha<0.5$. Lower bounds are presented to show that such constraints are necessary. The fourth contribution uses these results to show the learnability of DNF with membership queries under difference bounded tree BN. The final contribution is to develop an algorithm for learning difference-bounded tree BN distributions, thus extending the DNF learnability result to cases where the distribution is not known in advance.