Discretization Error of Fourier Neural Operators

TL;DR

This paper investigates the discretization error in Fourier Neural Operators, providing algebraic convergence rates and practical algorithms to optimize training by balancing discretization and model errors.

Contribution

The paper analyzes the discretization error in FNOs, deriving convergence rates and proposing an algorithm to optimize training efficiency based on error decomposition.

Findings

Discretization error converges algebraically with grid resolution.

Numerical experiments validate theoretical error rates.

An algorithm improves training efficiency by balancing errors.

Abstract

Operator learning is a variant of machine learning that is designed to approximate maps between function spaces from data. The Fourier Neural Operator (FNO) is one of the main model architectures used for operator learning. The FNO combines linear and nonlinear operations in physical space with linear operations in Fourier space, leading to a parameterized map acting between function spaces. Although in definition, FNOs are objects in continuous space and perform convolutions on a continuum, their implementation is a discretized object performing computations on a grid, allowing efficient implementation via the FFT. Thus, there is a discretization error between the continuum FNO definition and the discretized object used in practice that is separate from other previously analyzed sources of model error. We examine this discretization error here and obtain algebraic rates of convergence in terms of the grid resolution as a function of the input regularity. Numerical experiments that validate the theory and describe model stability are performed. In addition, an algorithm is presented that leverages the discretization error and model error decomposition to optimize computational training time.