When PINNs Go Wrong: Pseudo-Time Stepping Against Spurious Solutions

TL;DR

This paper investigates why physics-informed neural networks (PINNs) often fail or produce spurious solutions, and proposes an adaptive pseudo-time stepping method with collocation resampling to improve their reliability and accuracy.

Contribution

It reveals the fundamental weakness of residual loss in PINNs and introduces an adaptive pseudo-time stepping strategy that enhances robustness without extensive tuning.

Findings

Pseudo-time stepping combined with resampling helps detect and avoid spurious solutions.

The adaptive step size selection improves accuracy and robustness across PDE benchmarks.

The proposed method reduces the likelihood of convergence to physically incorrect solutions.

Abstract

Physics-informed neural networks (PINNs) provide a promising machine learning framework for solving partial differential equations, but their training often breaks down on challenging problems, sometimes converging to physically incorrect solutions despite achieving small residual losses. This failure, we argue, is not merely an optimization difficulty. Rather, it reflects a fundamental weakness of the empirical PDE residual loss, which can admit trivial or spurious solutions during training. From this perspective, we revisit pseudo-time stepping, a technique that has recently shown strong empirical success in PINNs. We show that its main benefit is not simply to ease optimization; instead, when combined with collocation-point resampling, it helps reveal and avoid spurious solutions. At the same time, we find that the effectiveness of pseudo-time stepping depends critically on the choice of step size, which cannot be tuned reliably from the training loss alone. To overcome this limitation, we propose an adaptive pseudo-time stepping strategy that selects the step size from a finite-difference surrogate of the local residual Jacobian, yielding the largest step permitted by local stability without per-problem tuning. Across a diverse set of PDE benchmarks, the proposed method consistently improves both accuracy and robustness. Together, these findings provide a clearer understanding of why PINNs fail and suggest a practical pathway toward more reliable physics-informed learning. All code and data accompanying this manuscript are available at https://github.com/sifanexisted/jaxpi2.