When and Why Adversarial Training Improves PINNs: A Neural Tangent Kernel Perspective

TL;DR

This paper offers a theoretical framework explaining when and why adversarial training enhances PINNs, introduces a new training algorithm, and demonstrates significant accuracy improvements over existing methods.

Contribution

It provides a novel analysis based on neural tangent kernels, clarifies the effectiveness of adversarial training in PINNs, and proposes an efficient training algorithm.

Findings

Adversarial training significantly reduces PINNs training pathology.

The proposed method achieves several magnitudes higher accuracy.

The framework offers theoretical insights into GAN-based PINNs training.

Abstract

Physics-informed neural networks (PINNs) are powerful surrogates for differential equations but are notoriously difficult to train due to spectral bias, stiffness, and poor accuracy on high-frequency or multiscale solutions. Adversarial training based on generative adversarial networks (GANs) has recently gained surprisingly strong empirical results in improving training, but the underlying mechanisms remain elusive. To this end, we propose a new analysis framework for adversarially trained PINNs, based on the key observation of how the discriminator in GANs can influence the training dynamics of PINNs. The framework first provides a much needed theoretical grounding to why and when adversarial training is effective in PINNs, then presents a unified analysis of GANs variants in such training, and finally leads to a new, practical, efficient training algorithm for PINNs. Empirical results demonstrate that our method can significantly reduce the pathology of PINNs training, thereby providing better models with superior performances, often several magnitudes more accurate than alternative methods.