The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures

TL;DR

This paper investigates the theoretical limits of learning Lipschitz operators in infinite-dimensional spaces under Gaussian measures, revealing fundamental sample complexity bounds and conditions for convergence.

Contribution

It provides the first tight characterization of sample complexity for Lipschitz operator learning, establishing lower and upper bounds and identifying conditions for near-algebraic convergence.

Findings

Sample complexity bounds are tightly characterized.

No method can achieve algebraic convergence with linear samples.

Spectral decay of covariance enables near-algebraic convergence.

Abstract

Operator learning, the approximation of mappings between infinite-dimensional function spaces using machine learning, has gained increasing research attention in recent years. Approximate operators, learned from data, can serve as efficient surrogate models for problems in computational science and engineering, complementing traditional methods. However, despite their empirical success, our understanding of the underlying mathematical theory is in large part still incomplete. In this paper, we study the approximation of Lipschitz operators with respect to Gaussian measures. We prove higher Gaussian Sobolev regularity of Lipschitz operators and establish lower and upper bounds on the Hermite polynomial approximation error. We then study general reconstruction strategies of Lipschitz operators from $m$ arbitrary (potentially adaptive) linear samples. As a key finding, we tightly characterize the corresponding sample complexity, that is, the smallest achievable worst-case error among all possible choices of (adaptive) sampling and reconstruction strategies in terms of $m$. As a consequence, we identify an inherent curse of sample complexity: No method to approximate Lipschitz operators based on $m$ linear samples can achieve algebraic convergence rates in $m$. On the positive side, we prove that a sufficiently fast spectral decay of the covariance operator of the underlying Gaussian measure guarantees convergence rates which are arbitrarily close to any algebraic rate. Overall, by tightly characterizing the sample complexity, our work confirms the intrinsic difficulty of learning Lipschitz operators, regardless of the data or learning technique.