TL;DR
This paper applies effective field theory to Fourier Neural Operators, providing a principled understanding of their frequency behavior, stability, and generalization, and introduces a calibration method that improves performance on PDE benchmarks.
Contribution
It offers a novel theoretical framework for analyzing FNOs, deriving conditions for stability and feature learning, and proposes a practical initialization method that enhances FNO training.
Findings
Nonlinear activations couple frequency inputs to high frequency modes.
Criticality conditions ensure stable propagation of perturbations in wide networks.
Calibrated FNOs outperform vanilla FNOs in stability and accuracy on PDE benchmarks.
Abstract
Fourier Neural Operators (FNOs) have emerged as leading surrogates for solver operators for various functional problems, yet their stability, generalization and frequency behavior lack a principled explanation. We present a systematic effective field theory analysis of FNOs in an infinite-dimensional function space, deriving closed recursion relations for the layer kernel and four-point vertex and then examining three practically important settings-analytic activations, scale-invariant cases and architectures with residual connections. The theory shows that nonlinear activations inevitably couple frequency inputs to high frequency modes that are otherwise discarded by spectral truncation, and experiments confirm this frequency transfer. For wide networks, we derive explicit criticality conditions on the weight initialization ensemble that ensure small input perturbations maintain a…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
1) Extends effective field theory to FNOs with explicit layerwise kernel/vertex recursions and susceptibility-based criticality conditions. 2) Provides a clear, principled explanation of frequency coupling beyond truncation and analyzes the roles of activation class and residual connections. 3) Technically careful theory with closed-form derivations and internal consistency checks that align with controlled simulations.
1). Section 4 states the three cases of analytic activations, scale invariant activations, and residual FNOs mostly as closed form recursions without interpretation or usable guidance. The text does not identify which convolution orders dominate low frequency versus high frequency behavior, when kernels grow or decay across depth, or how spectra leak beyond the truncation band. Assumptions on activation coefficients, on the ranges of $\alpha$ and $\beta$, and on residual strength $\gamma$ are no
1. **Solid Theoretical Analysis**: The paper's theoretical analysis is solid and rigorous. The authors successfully extend the Effective Field Theory (EFT) framework to analyze Fourier Neural Operators (FNOs) in infinite-dimensional function spaces. 2. **Validation of Theory with Experiments**: The authors do not merely present theory but also provide convincing numerical experiments that validate its reliability. The strong consistency between theoretical predictions and empirical results is
1. **Poor Writing and Presentation**: The paper's clarity is a significant issue, to the point of being catastrophic. The dense presentation of mathematical formulas, often without sufficient qualitative explanation, makes it exceedingly difficult for readers, who are unfamiliar with effective field theory, to grasp the core concepts and contributions. The appendix is also problematic, like Appendix H with experimental parameters presented in a long, unstructured format that hinders comprehensio
I am not necesseraly familiar with Effective Field Theory or even Neural Operators, but the authors do a pretty good job explaining the many notions involved. This paper essentially extends the NNGP analysis for DNNs to Neural Operators, which is novel to my knowledge. The NNGP analysis played a key role in later proving convergence through the NTK and we can hope that this could pave the way to a NTK type convergence result for Neural Operators. It is interesting to see a similar structure eme
The formulas are quite big and it can be difficult to get a high level intuition for how these covariances are evolving. Also it would be nice to compare to the NNGP litterature to see if there is any fundamental difference in the Neural Operator settings or whether it is just "the same but in infinite dimension". The results are restricted to the white noise / uncorrelated Fourier modes case, which kills all correlations between frequencies or spatial correlations (or at least that is what I c
1. Trying to use EFT to understand FNOs is a theoretically ambitious and interesting goal. 2. The paper shows a serious effort to derive the mathematical formulas carefully, which will be appreciated by theory-oriented readers.
1. **Unclear Motivation (Major):** The paper's biggest problem is that it doesn't make a strong case for why we need this complex EFT analysis for FNOs. It vaguely says that FNOs lack a "principled explanation" for their behavior, but it doesn't point to a specific, important problem in the existing FNO literature that this theory solves. Why is EFT the right tool for this job? The contribution feels disconnected from practical machine learning challenges. 2. **Hard to Read (Major):**
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
