Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators

TL;DR

This paper develops a theoretical framework for analyzing the generalization of neural operators using operator-valued kernels and random features, providing insights into learning rates and neural network requirements.

Contribution

It extends random feature analysis to operator-valued kernels, unifying kernel methods and neural operators within a rigorous theoretical setting.

Findings

Established optimal learning rates for neural operators.

Derived minimax rates in well-specified and misspecified cases.

Provided a unified analysis framework for spectral regularization methods.

Abstract

In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.