List Learning with Attribute Noise

TL;DR

This paper extends the model of attribute noise learning to list learning, demonstrating efficient learning of sparse conjunctions but proving the impossibility of efficiently learning parities and majorities.

Contribution

It introduces list learning with attribute noise, connecting it to coding theory, and provides both positive and negative results on learnability of different concepts.

Findings

Sparse conjunctions can be efficiently list learned under certain conditions.

Efficient learning of parities and majorities remains impossible in this model.

The model extends previous attribute noise learning frameworks with list decoding concepts.

Abstract

We introduce and study the model of list learning with attribute noise. Learning with attribute noise was introduced by Shackelford and Volper (COLT 1988) as a variant of PAC learning, in which the algorithm has access to noisy examples and uncorrupted labels, and the goal is to recover an accurate hypothesis. Sloan (COLT 1988) and Goldman and Sloan (Algorithmica 1995) discovered information-theoretic limits to learning in this model, which have impeded further progress. In this article we extend the model to that of list learning, drawing inspiration from the list-decoding model in coding theory, and its recent variant studied in the context of learning. On the positive side, we show that sparse conjunctions can be efficiently list learned under some assumptions on the underlying ground-truth distribution. On the negative side, our results show that even in the list-learning model, efficient learning of parities and majorities is not possible regardless of the representation used.