Multi-Preconditioned LBFGS for Training Finite-Basis PINNs

TL;DR

This paper introduces a multi-preconditioned LBFGS algorithm for efficiently training finite-basis physics-informed neural networks, leveraging domain decomposition and local corrections to enhance convergence and accuracy.

Contribution

The paper proposes a novel multi-preconditioning mechanism for LBFGS, tailored for FBPINNs, improving training speed and accuracy with reduced communication overhead.

Findings

MP-LBFGS accelerates convergence compared to standard LBFGS.

The method improves model accuracy in numerical experiments.

Lower communication overhead than existing approaches.

Abstract

A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear additive Schwarz method and exploits the domain-decomposition-inspired additive architecture of FBPINNs, in which local neural networks are defined on subdomains, thereby localizing the network representation. Parallel, subdomain-local quasi-Newton corrections are then constructed on the corresponding local parts of the architecture. A key feature is a novel nonlinear multi-preconditioning mechanism, in which subdomain corrections are optimally combined through the solution of a low-dimensional subspace minimization problem. Numerical experiments indicate that MP-LBFGS can improve convergence speed, as well as model accuracy over standard LBFGS while incurring lower communication overhead.