On Generation in Metric Spaces

TL;DR

This paper explores the concept of generation in separable metric spaces, introducing a new scale-sensitive dimension and analyzing how generatability varies across different types of metric spaces, including finite and infinite-dimensional ones.

Contribution

It extends the framework of language generation to general metric spaces, introduces the $( ext{}ackslashvarepsilon, ext{}ackslashvarepsilon' ext{)}$-closure dimension, and characterizes stability and variability of generatability across different spaces.

Findings

Generatability is stable in doubling spaces and invariant under metric equivalence.

In general metric spaces, generatability can be highly scale-sensitive.

In infinite-dimensional Hilbert spaces, all notions of generation may fail with parameter changes.

Abstract

We study generation in separable metric instance spaces. We extend the language generation framework from Kleinberg and Mullainathan [2024] beyond countable domains by defining novelty through metric separation and allowing asymmetric novelty parameters for the adversary and the generator. We introduce the $(\varepsilon,\varepsilon')$-closure dimension, a scale-sensitive analogue of closure dimension, which yields characterizations of uniform and non-uniform generatability and a sufficient condition for generation in the limit. Along the way, we identify a sharp geometric contrast. Namely, in doubling spaces, including all finite-dimensional normed spaces, generatability is stable across novelty scales and invariant under equivalent metrics. In general metric spaces, however, generatability can be highly scale-sensitive and metric-dependent; even in the natural infinite-dimensional Hilbert space $\ell^2$, all notions of generation may fail abruptly as the novelty parameters vary.