Minimax Optimal Kernel Operator Learning via Multilevel Training

TL;DR

This paper investigates the fundamental limits of learning Hilbert-Schmidt operators between infinite-dimensional Sobolev spaces and introduces a multilevel training algorithm that achieves optimal learning rates.

Contribution

It establishes the information-theoretic lower bounds for operator learning and proposes a multilevel spectral regularization method that attains these bounds.

Findings

Theoretical lower bounds for Sobolev Hilbert-Schmidt operator learning.

A multilevel spectral regularization algorithm matching the optimal rates.

Flexibility in algorithm design through spectral component selection.

Abstract

Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning. In this paper, we study the statistical limit of learning a Hilbert-Schmidt operator between two infinite-dimensional Sobolev reproducing kernel Hilbert spaces. We establish the information-theoretic lower bound in terms of the Sobolev Hilbert-Schmidt norm and show that a regularization that learns the spectral components below the bias contour and ignores the ones that are above the variance contour can achieve the optimal learning rate. At the same time, the spectral components between the bias and variance contours give us flexibility in designing computationally feasible machine learning algorithms. Based on this observation, we develop a multilevel kernel operator learning algorithm that is optimal when learning linear operators between infinite-dimensional function spaces.