Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs

TL;DR

This paper introduces an adversarial adaptive sampling method that combines PINNs and optimal transport to improve neural network solutions of PDEs by optimizing both the solution and training samples simultaneously.

Contribution

It proposes a novel minmax formulation that jointly optimizes the neural network solution and the training samples using a deep generative model and Wasserstein distance.

Findings

Reduces variance of residuals for better PDE approximation

Significantly decreases Monte Carlo error in loss functional

First to unify residual minimization and sample optimization in one framework

Abstract

Solving partial differential equations (PDEs) is a central task in scientific computing. Recently, neural network approximation of PDEs has received increasing attention due to its flexible meshless discretization and its potential for high-dimensional problems. One fundamental numerical difficulty is that random samples in the training set introduce statistical errors into the discretization of loss functional which may become the dominant error in the final approximation, and therefore overshadow the modeling capability of the neural network. In this work, we propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set, provided by a deep generative model. The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile when it is being minimized. Such an idea is achieved by implicitly embedding the Wasserstein distance between the residual-induced distribution and the uniform distribution into the loss, which is then minimized together with the residual. A nearly uniform residual profile means that its variance is small for any normalized weight function such that the Monte Carlo approximation error of the loss functional is reduced significantly for a certain sample size. The adversarial adaptive sampling (AAS) approach proposed in this work is the first attempt to formulate two essential components, minimizing the residual and seeking the optimal training set, into one minmax objective functional for the neural network approximation of PDEs.