The Ill-Posed Foundations of Physics-Informed Neural Networks and Their Finite-Difference Variants

TL;DR

This paper reveals that both AD- and FD-PINNs are inherently ill-posed for solving PDEs, but FD-PINNs exhibit more stable behavior and robustness, explained through a unified mathematical framework.

Contribution

It provides a unified mathematical foundation for AD- and FD-PINNs, proving their ill-posedness and elucidating the structural reasons behind FD-PINNs' stability.

Findings

Both AD- and FD-PINNs are ill-posed with non-unique minimizers.

FD-PINNs are tightly coupled to finite-difference schemes and align with discrete PDE solutions.

FD-PINNs demonstrate stability and robustness in forward and inverse problems, even with noisy data.

Abstract

Physics-informed neural networks based on automatic differentiation (AD-PINNs) and their finite-difference counterparts (FD-PINNs) are widely used for solving partial differential equations (PDEs), yet their analytical properties remain poorly understood. This work provides a unified mathematical foundation for both formulations. Under mild regularity assumptions on the activation function and for sufficiently wide neural networks of depth at least two, we prove that both the AD- and FD-PINN optimization problems are ill-posed: whenever a minimizer exists, there are in fact infinitely many, and uniqueness fails regardless of the choice of collocation points or finite-difference stencil. Nevertheless, we establish two structural properties. First, whenever the underlying PDE or its finite-difference discretization admits a solution, the corresponding AD-PINN or FD-PINN loss also admits a minimizer, realizable by a neural network of finite width. Second, FD-PINNs are tightly coupled to the underlying finite-difference scheme: every FD-PINN minimizer agrees with a finite-difference minimizer on the grid, and in regimes where the discrete PDE solution is unique, all zero-loss FD-PINN minimizers coincide with the discrete PDE solution on the stencil. Numerical experiments illustrate these theoretical insights: FD-PINNs remain stable in representative forward and inverse problems, including settings where AD-PINNs may fail to converge. We also include an inverse problem with noisy data, demonstrating that FD-PINNs retain robustness in this setting as well. Taken together, our results clarify the analytical limitations of AD-PINNs and explain the structural reasons for the more stable behavior observed in FD-PINNs.