On Learning Thresholds of Parities and Unions of Rectangles in Random Walk Models

TL;DR

This paper explores the learnability of thresholds of parities and unions of rectangles in various random walk models, showing limitations in some models and efficient algorithms in others, extending previous work on DNFs.

Contribution

It demonstrates the non-learnability of thresholds of parities in the Noise Sensitivity model and provides efficient learning algorithms in a cyclic Random Walk model, also extending to unions of rectangles.

Findings

Thresholds of parities cannot be learned efficiently in the Noise Sensitivity model.

Efficient learning of polynomially weighted thresholds of parities is possible in a cyclic Random Walk model.

The algorithm for DNFs is extended to unions of rectangles over larger domains.

Abstract

In a recent breakthrough, [Bshouty et al., 2005] obtained the first passive-learning algorithm for DNFs under the uniform distribution. They showed that DNFs are learnable in the Random Walk and Noise Sensitivity models. We extend their results in several directions. We first show that thresholds of parities, a natural class encompassing DNFs, cannot be learned efficiently in the Noise Sensitivity model using only statistical queries. In contrast, we show that a cyclic version of the Random Walk model allows to learn efficiently polynomially weighted thresholds of parities. We also extend the algorithm of Bshouty et al. to the case of Unions of Rectangles, a natural generalization of DNFs to $\{0,...,b-1\}^n$.