FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks

TL;DR

This paper reveals that the failure modes in Physics-Informed Neural Networks are primarily caused by insufficient numerical precision, and upgrading to FP64 precision can reliably solve PDEs without failure.

Contribution

The study demonstrates that increasing arithmetic precision from FP32 to FP64 resolves PINN failure modes, challenging previous explanations based on local optima.

Findings

FP64 precision enables successful PDE solving with PINNs.

Failure modes are due to numerical precision limits, not local minima.

Training dynamics shift with numerical precision, affecting convergence.

Abstract

Physics Informed Neural Networks (PINNs) often exhibit failure modes in which the PDE residual loss converges while the solution error stays large, a phenomenon traditionally blamed on local optima separated from the true solution by steep loss barriers. We challenge this understanding by demonstrate that the real culprit is insufficient arithmetic precision: with standard FP32, the LBFGS optimizer prematurely satisfies its convergence test, freezing the network in a spurious failure phase. Simply upgrading to FP64 rescues optimization, enabling vanilla PINNs to solve PDEs without any failure modes. These results reframe PINN failure modes as precision induced stalls rather than inescapable local minima and expose a three stage training dynamic unconverged, failure, success whose boundaries shift with numerical precision. Our findings emphasize that rigorous arithmetic precision is the key to dependable PDE solving with neural networks.