Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations

TL;DR

This paper introduces a multi-task learning framework with adversarial training to enhance physics-informed neural networks, significantly improving their ability to solve complex, high-nonlinearity PDEs more accurately and generalize better across different problem domains.

Contribution

The paper proposes a novel multi-task learning approach combined with adversarial training to improve PDE-solving neural networks, especially in high nonlinearity regions, which is a new strategy in this context.

Findings

Reduces error on unseen data points in various PDE examples.

Improves generalization in high-dimensional stochastic PDEs.

Enhances learning in high nonlinearity domains.

Abstract

Recently, researchers have utilized neural networks to accurately solve partial differential equations (PDEs), enabling the mesh-free method for scientific computation. Unfortunately, the network performance drops when encountering a high nonlinearity domain. To improve the generalizability, we introduce the novel approach of employing multi-task learning techniques, the uncertainty-weighting loss and the gradients surgery, in the context of learning PDE solutions. The multi-task scheme exploits the benefits of learning shared representations, controlled by cross-stitch modules, between multiple related PDEs, which are obtainable by varying the PDE parameterization coefficients, to generalize better on the original PDE. Encouraging the network pay closer attention to the high nonlinearity domain regions that are more challenging to learn, we also propose adversarial training for generating supplementary high-loss samples, similarly distributed to the original training distribution. In the experiments, our proposed methods are found to be effective and reduce the error on the unseen data points as compared to the previous approaches in various PDE examples, including high-dimensional stochastic PDEs.