Generation from Noisy Examples

TL;DR

This paper extends the theoretical understanding of generation from examples to noisy settings, establishing conditions under which a hypothesis class can still generate unseen positive examples despite noise interference.

Contribution

It introduces necessary and sufficient conditions for noisy generatability of binary hypothesis classes, expanding prior noiseless results to account for noisy example streams.

Findings

Generatability largely unaffected by finite noisy examples for finite and countable classes.

Provides a framework for understanding generation under adversarial noise.

Extends previous noiseless generation results to noisy environments.

Abstract

We continue to study the learning-theoretic foundations of generation by extending the results from Kleinberg and Mullainathan [2024] and Li et al. [2024] to account for noisy example streams. In the noiseless setting of Kleinberg and Mullainathan [2024] and Li et al. [2024], an adversary picks a hypothesis from a binary hypothesis class and provides a generator with a sequence of its positive examples. The goal of the generator is to eventually output new, unseen positive examples. In the noisy setting, an adversary still picks a hypothesis and a sequence of its positive examples. But, before presenting the stream to the generator, the adversary inserts a finite number of negative examples. Unaware of which examples are noisy, the goal of the generator is to still eventually output new, unseen positive examples. In this paper, we provide necessary and sufficient conditions for when a binary hypothesis class can be noisily generatable. We provide such conditions with respect to various constraints on the number of distinct examples that need to be seen before perfect generation of positive examples. Interestingly, for finite and countable classes we show that generatability is largely unaffected by the presence of a finite number of noisy examples.