Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks

TL;DR

This paper introduces a lightweight, curvature-aware optimization method that enhances training efficiency and accuracy of Physics-Informed Neural Networks across various PDE benchmarks.

Contribution

It presents a plug-and-play, computationally efficient correction framework that improves optimizer performance without forming second-order matrices.

Findings

Consistent improvements in convergence speed and stability.

Enhanced solution accuracy on diverse PDE benchmarks.

Effective on high-dimensional and complex systems.

Abstract

Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of their loss landscapes. We propose a lightweight curvature-aware optimization framework that augments existing first-order optimizers with an adaptive predictive correction based on secant information. Consecutive gradient differences are used as a cheap proxy for local geometric change, together with a step-normalized secant curvature indicator to control the correction strength. The framework is plug-and-play, computationally efficient, and broadly compatible with existing optimizers, without explicitly forming second-order matrices. Experiments on diverse PDE benchmarks show consistent improvements in convergence speed, training stability, and solution accuracy over standard optimizers and strong baselines, including on the high-dimensional heat equation, Gray--Scott system, Belousov--Zhabotinsky system, and 2D Kuramoto--Sivashinsky system.