A Residual Guided strategy with Generative Adversarial Networks in training Physics-Informed Transformer Networks

TL;DR

This paper introduces a novel training strategy combining Transformers and GANs to improve physics-informed neural networks for solving complex PDEs, significantly reducing errors in critical regions.

Contribution

It proposes a residual-guided training framework with a causal penalty and adaptive sampling, enhancing accuracy and temporal causality in physics-informed Transformer networks.

Findings

Achieved up to three orders of magnitude reduction in relative MSE.

Effectively identified and prioritized high-residual regions during training.

Demonstrated improved performance on Allen-Cahn, Klein-Gordon, and Navier-Stokes equations.

Abstract

Nonlinear partial differential equations (PDEs) are pivotal in modeling complex physical systems, yet traditional Physics-Informed Neural Networks (PINNs) often struggle with unresolved residuals in critical spatiotemporal regions and violations of temporal causality. To address these limitations, we propose a novel Residual Guided Training strategy for Physics-Informed Transformer via Generative Adversarial Networks (GAN). Our framework integrates a decoder-only Transformer to inherently capture temporal correlations through autoregressive processing, coupled with a residual-aware GAN that dynamically identifies and prioritizes high-residual regions. By introducing a causal penalty term and an adaptive sampling mechanism, the method enforces temporal causality while refining accuracy in problematic domains. Extensive numerical experiments on the Allen-Cahn, Klein-Gordon, and Navier-Stokes equations demonstrate significant improvements, achieving relative MSE reductions of up to three orders of magnitude compared to baseline methods. This work bridges the gap between deep learning and physics-driven modeling, offering a robust solution for multiscale and time-dependent PDE systems.