Convergence Rates for Learning Linear Operators from Noisy Data

TL;DR

This paper analyzes the convergence rates for learning linear operators in infinite-dimensional Hilbert spaces from noisy data, providing theoretical guarantees and numerical validation for eigenvalue estimation under a Bayesian framework.

Contribution

It introduces a Bayesian approach to estimate operator eigenvalues, deriving convergence rates and generalization bounds in infinite-dimensional settings with noisy data.

Findings

Posterior contraction rates depend on data smoothness and eigenvalue decay.

Theoretical bounds are validated through numerical experiments.

Results apply to both compact and unbounded operators.

Abstract

This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear operator. Assuming that the operator is diagonalizable in a known basis, this work solves the equivalent inverse problem of estimating the operator's eigenvalues given the data. Adopting a Bayesian approach, the theoretical analysis establishes posterior contraction rates in the infinite data limit with Gaussian priors that are not directly linked to the forward map of the inverse problem. The main results also include learning-theoretic generalization error guarantees for a wide range of distribution shifts. These convergence rates quantify the effects of data smoothness and true eigenvalue decay or growth, for compact or unbounded operators, respectively, on sample complexity. Numerical evidence supports the theory in diagonal and non-diagonal settings.