Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies

TL;DR

This paper introduces domain-decomposition based nonlinear preconditioning strategies to improve the training efficiency and accuracy of physics-informed neural networks, leveraging Schwarz methods for enhanced convergence and parallelism.

Contribution

It presents novel nonlinear additive and multiplicative preconditioners for PINNs, utilizing Schwarz domain decomposition to significantly improve optimizer convergence and solution accuracy.

Findings

Preconditioners significantly improve L-BFGS convergence.

Additive preconditioner enables parallel training.

Enhanced solution accuracy for PDEs.

Abstract

We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.