Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks

TL;DR

This paper introduces a domain decomposition approach combined with Gauss-Newton optimization to improve the training efficiency of physics-informed neural networks for solving boundary value problems governed by PDEs.

Contribution

It proposes a novel combination of domain decomposition and Gauss-Newton method to accelerate training and handle spectral bias in physics-informed neural networks.

Findings

Faster convergence with Gauss-Newton compared to Adam.

Reduced computational cost due to block-sparse structure.

Promising results for PDE neural network solvers.

Abstract

Approximating the solutions of boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process. This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components, and can be mitigated by localizing neural networks via (overlapping) domain decomposition. We combine this localization with the Gauss-Newton method as the optimizer to obtain faster convergence than gradient-based schemes such as Adam; this comes at the cost of solving an ill-conditioned linear system in each iteration. Domain decomposition induces a block-sparse structure in the otherwise dense Gauss-Newton system, reducing the computational cost per iteration. Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.