Annealed adaptive importance sampling method in PINNs for solving high dimensional partial differential equations

TL;DR

This paper introduces Annealed Adaptive Importance Sampling (AAIS) to enhance PINNs by strategically sampling in high residual regions, significantly improving accuracy and efficiency in solving high-dimensional and singular PDEs.

Contribution

The paper proposes a novel AAIS method for PINNs that adaptively samples points to improve PDE residual approximation, especially in challenging high-dimensional and singular cases.

Findings

AAIS improves PINN efficiency across various PDEs.

Enhanced accuracy in high-dimensional PDE solutions.

Effective in handling singular PDE cases.

Abstract

Physics-informed neural networks (PINNs) have emerged as powerful tools for solving a wide range of partial differential equations (PDEs). However, despite their user-friendly interface and broad applicability, PINNs encounter challenges in accurately resolving PDEs, especially when dealing with singular cases that may lead to unsatisfactory local minima. To address these challenges and improve solution accuracy, we propose an innovative approach called Annealed Adaptive Importance Sampling (AAIS) for computing the discretized PDE residuals of the cost functions, inspired by the Expectation Maximization algorithm used in finite mixtures to mimic target density. Our objective is to approximate discretized PDE residuals by strategically sampling additional points in regions with elevated residuals, thus enhancing the effectiveness and accuracy of PINNs. Implemented together with a straightforward resampling strategy within PINNs, our AAIS algorithm demonstrates significant improvements in efficiency across a range of tested PDEs, even with limited training datasets. Moreover, our proposed AAIS-PINN method shows promising capabilities in solving high-dimensional singular PDEs. The adaptive sampling framework introduced here can be integrated into various PINN frameworks.