Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes

TL;DR

This paper introduces Neural Galerkin schemes that use adaptive particle-based ensembles to efficiently estimate training loss for neural network solutions of PDEs, especially in complex, high-dimensional problems.

Contribution

It proposes a novel coupling of particle dynamics with neural parametrizations to adaptively sample data, improving loss estimation in PDE training.

Findings

Few particles suffice for accurate loss estimation.

Adaptive sampling improves efficiency in high-dimensional PDEs.

Method handles local features like waves effectively.

Abstract

Training nonlinear parametrizations such as deep neural networks to numerically approximate solutions of partial differential equations is often based on minimizing a loss that includes the residual, which is analytically available in limited settings only. At the same time, empirically estimating the training loss is challenging because residuals and related quantities can have high variance, especially for transport-dominated and high-dimensional problems that exhibit local features such as waves and coherent structures. Thus, estimators based on data samples from un-informed, uniform distributions are inefficient. This work introduces Neural Galerkin schemes that estimate the training loss with data from adaptive distributions, which are empirically represented via ensembles of particles. The ensembles are actively adapted by evolving the particles with dynamics coupled to the nonlinear parametrizations of the solution fields so that the ensembles remain informative for estimating the training loss. Numerical experiments indicate that few dynamic particles are sufficient for obtaining accurate empirical estimates of the training loss, even for problems with local features and with high-dimensional spatial domains.