Extending Neural Operators: Robust Handling of Functions Beyond the Training Set

TL;DR

This paper introduces a theoretical framework for extending neural operators to effectively handle out-of-distribution functions, leveraging kernel methods and RKHS theory, with empirical validation on PDEs involving complex geometries.

Contribution

It develops a rigorous RKHS-based framework for neural operator extension, including theorems on approximation accuracy and relationships between kernels and Sobolev spaces.

Findings

Theoretical conditions for reliable neural operator extension.

Empirical validation on PDEs with manifold and point-cloud data.

Analysis of factors affecting accuracy and computational efficiency.

Abstract

We develop a rigorous framework for extending neural operators to handle out-of-distribution input functions. We leverage kernel approximation techniques and provide theory for characterizing the input-output function spaces in terms of Reproducing Kernel Hilbert Spaces (RKHSs). We provide theorems on the requirements for reliable extensions and their predicted approximation accuracy. We also establish formal relationships between specific kernel choices and their corresponding Sobolev Native Spaces. This connection further allows the extended neural operators to reliably capture not only function values but also their derivatives. Our methods are empirically validated through the solution of elliptic partial differential equations (PDEs) involving operators on manifolds having point-cloud representations and handling geometric contributions. We report results on key factors impacting the accuracy and computational performance of the extension approaches.