Resolving Sharp Gradients of Unstable Singularities to Machine Precision via Neural Networks

TL;DR

This paper introduces a gradient-normalized residual re-weighting scheme in neural networks to accurately resolve highly unstable singularities in fluid dynamics equations, achieving near machine precision and discovering new unstable solutions.

Contribution

The work presents a novel gradient-normalized residual re-weighting method combined with multi-stage neural networks to accurately identify and discover highly unstable singularities in nonlinear PDEs.

Findings

Achieved near machine precision for high-gradient solutions

Discovered new unstable solutions for IPM and Schrödinger equations

Enabled high-precision solutions for previously intractable singularities

Abstract

Recent work introduced a robust computational framework combining embedded mathematical structures, advanced optimization, and neural network architecture, leading to the discovery of multiple unstable self-similar solutions for key fluid dynamics equations, including the Incompressible Porous Media (IPM) and 2D Boussinesq systems. While this framework confirmed the existence of these singularities, an accuracy level approaching double-float machine precision was only achieved for stable and 1st unstable solutions of the 1D C\'ordoba-C\'ordoba-Fontelos model. For highly unstable solutions characterized by extreme gradients, the accuracy remained insufficient for validation. The primary obstacle is the presence of sharp solution gradients. Those gradients tend to induce large, localized PDE residuals during training, which not only hinder convergence, but also obscure the subtle signals near the origin required to identify the correct self-similar scaling parameter lambda of the solutions. In this work, we introduce a gradient-normalized PDE residual re-weighting scheme to resolve the high-gradient challenge while amplifying the critical residual signals at the origin for lambda identification. Coupled with the multi-stage neural network architecture, the PDE residuals are reduced to the level of round-off error across a wide spectrum of unstable self-similar singularities previously discovered. Furthermore, our method enables the discovery of new highly unstable singularities, i.e. the 4th unstable solution for IPM equations and a novel family of highly unstable solitons for the Nonlinear Schr\"odinger equations. This results in achieving high-gradient solutions with high precision, providing an important ingredient for bridging the gap between numerical discovery and computer-assisted proofs for unstable phenomena in nonlinear PDEs.