Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features
Yulun Wu, Miguel Aguiar, Karl H.Johansson, Matthieu Barreau
TL;DR
This paper introduces IFeF-PINN, an iterative training algorithm for physics-informed neural networks that uses Fourier-enhanced features to better learn high-frequency components and overcome spectral bias, with proven convergence and superior performance.
Contribution
The paper proposes a novel iterative training scheme for PINNs that incorporates Random Fourier Features to enhance high-frequency learning capabilities.
Findings
Enhanced approximation of high-frequency PDEs.
Proven convergence of the iterative training scheme.
Superior performance over existing algorithms on benchmark problems.
Abstract
Spectral bias, the tendency of neural networks to learn low-frequency features first, is a well-known issue with many training algorithms for physics-informed neural networks (PINNs). To overcome this issue, we propose IFeF-PINN, an algorithm for iterative training of PINNs with Fourier-enhanced features. The key idea is to enrich the latent space using high-frequency components through Random Fourier Features. This creates a two-stage training problem: (i) estimate a basis in the feature space, and (ii) perform regression to determine the coefficients of the enhanced basis functions. For an underlying linear model, it is shown that the latter problem is convex, and we prove that the iterative training scheme converges. Furthermore, we empirically establish that Random Fourier Features enhance the expressive capacity of the network, enabling accurate approximation of high-frequency…
Peer Reviews
Decision·ICLR 2026 Poster
The paper excels in its clear, well-motivated algorithmic innovation and solid theoretical–empirical balance. First, it provides a principled bi-level reformulation of PINN training that explicitly decouples feature learning from coefficient regression. This structure not only clarifies optimization dynamics but also leads to provable convexity and convergence for linear PDEs, a strong theoretical contribution. Second, the integration of Random Fourier Features into the latent space is cleve
A clear weakness of the paper is its limited treatment of nonlinear PDEs, where the proposed convex lower-level formulation breaks down. While the authors acknowledge that the lower regression problem becomes nonconvex in such cases (and suggest it to be promising direction), they did not offer quick remedies or convergence guarantee beyond gradient descent heuristics. This has limited its applicability as many practical PDEs of interest, such as fluid dynamics, reaction–diffusion systems, nonli
1. The paper addresses a well-known and important problem of spectral bias in PINNs. 2. The proposed approach is conceptually general and can be integrated with various PINN frameworks. 3. Provides theoretical convergence analysis and demonstrates improvements on canonical problems.
1. The paper is unclear in how the nominal and extended bases are created or selected, and lacks an explanation for the motivation behind modifying the last layer to improve spectral properties. The presentation of the methodology could be improved. 2. The discussion of the bi-level optimization framework is limited, and different possible formulations and the rationale for the chosen structure are not justified. 3. Computational aspects, such as training cost, convergence behavior, and initia
* (Clarity) The paper is easy to follow. Contributions are well clarified and compared with other studies addressing the spectral bias. * (Technical contribution) The proposed methods effectively combines PINNs and RFFs in a novel way, addressing the limited approximation ability of RFFs in low dimensions. * (Broader impact & Technical contribution) An application to general neural architectures is straightforward, with some careful treatments described in Remark 2 in Section 2. * (Theoretical c
- Although the theoretical convergence is proved, the proposed method requires a bilevel optimization, which may cause training instability, necessitating warm-start training outlined in Section 3 (please correct me if I missed singing); that said, pretraining or warm-start is often required when training PINN's variants. - Currently, the code is not available, although it will be released upon acceptance (line 353). Could the authors provide the code? Just a snippet is acceptable. ## Review s
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel Reduction and Neural Networks · Machine Learning in Materials Science · Gaussian Processes and Bayesian Inference