Derivative-Informed Fourier Neural Operator: Universal Approximation and Applications to PDE-Constrained Optimization

TL;DR

This paper introduces derivative-informed Fourier neural operators (DIFNOs) that can accurately learn operators and their derivatives, enabling efficient PDE-constrained optimization with theoretical guarantees and practical efficiency improvements.

Contribution

The paper develops a new derivative-informed training method for Fourier neural operators, providing universal approximation guarantees for operators and their derivatives, and demonstrates improved efficiency in PDE applications.

Findings

DIFNOs closely emulate operators and their sensitivities.

Theoretical universal approximation of operators and derivatives by FNOs.

Numerical results show superior sample efficiency and accuracy in PDE problems.

Abstract

We present approximation theories and efficient training methods for derivative-informed Fourier neural operators (DIFNOs) with applications to PDE-constrained optimization. A DIFNO is an FNO trained by minimizing its prediction error jointly on output and Fr\'echet derivative samples of a high-fidelity operator (e.g., a parametric PDE solution operator). As a result, a DIFNO can closely emulate not only the high-fidelity operator's response but also its sensitivities. To motivate the use of DIFNOs instead of conventional FNOs as surrogate models, we show that accurate surrogate-driven PDE-constrained optimization requires accurate surrogate Fr\'echet derivatives. Then, we establish (i) simultaneous universal approximation of continuously differentiable operators and their Fr\'echet derivatives by FNOs on compact sets, and (ii) universal approximation of continuously differentiable operators by FNOs in weighted Sobolev spaces with input measures that have unbounded supports. Our theoretical results certify the capability of FNOs for accurate derivative-informed operator learning and for the solution of PDE-constrained optimization problems. Furthermore, we develop efficient training schemes that leverage dimensionality reduction and multi-resolution techniques to significantly reduce memory and computational costs in Fr\'echet derivative learning. Numerical examples on nonlinear diffusion--reaction, Helmholtz, and Navier--Stokes equations demonstrate that DIFNOs are superior in sample complexity for operator learning and solving infinite-dimensional PDE-constrained inverse problems, achieving high accuracy at low training sample sizes.