Performance of Neural and Polynomial Operator Surrogates

TL;DR

This paper empirically compares neural and polynomial surrogate methods for parametric PDEs, analyzing their efficiency and accuracy across different input regularities and problem complexities.

Contribution

It provides a systematic evaluation of various surrogate models, highlighting their strengths and limitations depending on input smoothness and computational costs.

Findings

Polynomial surrogates excel for smooth inputs ($s \,\geq\, 2$).

Fourier neural operator converges fastest for rough inputs ($s \,\leq\, 1$).

Derivative-informed training improves data efficiency, especially for rough inputs.

Abstract

We consider the problem of constructing surrogate operators for parameter-to-solution maps arising from parametric partial differential equations, where repeated forward model evaluations are computationally expensive. We present a systematic empirical comparison of neural operator surrogates, including a reduced-basis neural operator trained with $L^2_\mu$ and $H^1_\mu$ objectives and the Fourier neural operator, against polynomial surrogate methods, specifically a reduced-basis sparse-grid surrogate and a reduced-basis tensor-train surrogate. All methods are evaluated on a linear parametric diffusion problem and a nonlinear parametric hyperelasticity problem, using input fields with algebraically decaying spectral coefficients at varying rates of decay $s$. To enable fair comparisons, we analyze ensembles of surrogate models generated by varying hyperparameters and compare the resulting Pareto frontiers of cost versus approximation accuracy, decomposing cost into contributions from data generation, setup, and evaluation. Our results show that no single method is universally superior. Polynomial surrogates achieve substantially better data efficiency for smooth input fields ($s \geq 2$), with convergence rates for the sparse-grid surrogate in agreement with theoretical predictions. For rough inputs ($s \leq 1$), the Fourier neural operator displays the fastest convergence rates. Derivative-informed training consistently improves data efficiency over standard $L^2_\mu$ training, providing a competitive alternative for rough inputs in the low-data regime when Jacobian information is available at reasonable cost. These findings highlight the importance of matching the surrogate methodology to the regularity of the problem as well as accuracy demands and computational constraints of the application.