Stability in Training PINNs for Stiff PDEs: Why Initial Conditions Matter
Baoli Hao, Ulisses Braga-Neto, Chun Liu, Lifan Wang, Ming Zhong
TL;DR
This paper reveals that enforcing initial conditions exactly is crucial for stable and efficient training of Physics-Informed Neural Networks on stiff PDEs, highlighting a key factor often overlooked.
Contribution
It systematically analyzes the impact of initial condition enforcement strategies on PINN stability, establishing their essential role in stiff PDE problems.
Findings
Exact enforcement of initial conditions is necessary for stability.
Adaptive loss weighting alone is insufficient without proper IC enforcement.
PINNs perform reliably when initial conditions are strictly enforced.
Abstract
Training Physics-Informed Neural Networks (PINNs) on stiff time-dependent PDEs remains highly unstable. Through rigorous ablation studies, we identify a surprisingly critical factor: the enforcement of initial conditions. We present the first systematic ablation of two core strategies, hard initial-condition constraints and adaptive loss weighting. Across challenging benchmarks (sharp transitions, higher-order derivatives, coupled systems, and high frequency modes), we find that exact enforcement of initial conditions (ICs) is not optional but essential. Our study demonstrates that stability and efficiency in PINN training fundamentally depend on ICs, paving the way toward more reliable PINN solvers in stiff regimes.
Peer Reviews
Decision·Submitted to ICLR 2026
* The paper tackles an important problem in PINN training, namely the hard enforcement of initial/boundary conditions with a simple and easy to implement modification of the models. * Numerical results show that there is an improvement in error using HC-PINNs. * The technique extends to standard IC/BC as well as periodic BCs. * Theoretical results on NTK conditioning are provided.
* Numerical evaluation is done only on 2D settings. * While the final learned solutions have improved error, they are still far from machine precision, which is more important and relevant for PDE applications. * Experimental tuning is limited (I believe only one architecture is studied without much tuning of optimization parameters). * Presentation and exposition is very poor. For example: - Line 200 does not make sense, $\psi$ is both 0 and $u_0$. No conditions are given on $\phi$. This is
S1) Along with experimental results a theoretical analysis of the hard constraint formulation using Neural Tangent Kernel (NTK) is provided. This enables the PINN to capture the exact ICs and hence helps reduce the spectral bias. S2) The proposed technique can be combined with other existing advanced PINN variants such as time marching PINNs, causal PINNs, RBA PINNs and curriculum training. S3) Detailed experimental evidence on a wide range of PDEs including comparison with other PINN variants
W1) Detailed ablations and explanations on the choice of the decay rate $C$ have not been included. Two specific values for the decay rate have been used. W2) As per my understanding only periodic boundary conditions have been studied, how does this framework extend to other types of boundary conditions. Is this a limitation or only periodic conditions have been chosen to be studied in this work.
- The paper investigates an important problem, enforcing hard constraints, which is of general interest to the PINNs community. While enforcement of hard constraints is a techniques previously implemented across the PINN literature, a thorough ablation study of the effectiveness of this technique has not been previously conducted, to my knowledge. - The experiments are generally thorough, with evaluations on seven stiff PDE systems and detailed ablations in the appendix. - The method is supporte
- I'm slightly concerned about the novelty of the contribution. I believe the rigorous ablation studies of hard constraint enforcement is a novel and important contribution for the PINN community. However, the work also seems to claim their HC-PINN (specifically their approach as defined in Section 2) as a novel contribution. These techniques have been present in the literature, for example see the reparameterizations in [1, 2, 3]. I believe the work could benefit from clarifying that the main c
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNumerical methods for differential equations