ANaGRAM: A Natural Gradient Relative to Adapted Model for efficient PINNs learning

TL;DR

This paper introduces a natural gradient method tailored for PINNs, significantly enhancing training speed and accuracy by leveraging differential geometry and Green's function theory.

Contribution

It presents a novel natural gradient algorithm for PINNs with scalable complexity and a principled reformulation of the PINNs problem grounded in geometric and Green's function analysis.

Findings

Improved training speed and accuracy of PINNs.

Scalable natural gradient algorithm with complexity depending on parameters and batch size.

Theoretical connection between PINNs reformulation and Green's functions.

Abstract

In the recent years, Physics Informed Neural Networks (PINNs) have received strong interest as a method to solve PDE driven systems, in particular for data assimilation purpose. This method is still in its infancy, with many shortcomings and failures that remain not properly understood. In this paper we propose a natural gradient approach to PINNs which contributes to speed-up and improve the accuracy of the training. Based on an in depth analysis of the differential geometric structures of the problem, we come up with two distinct contributions: (i) a new natural gradient algorithm that scales as $\min(P^2S, S^2P)$, where $P$ is the number of parameters, and $S$ the batch size; (ii) a mathematically principled reformulation of the PINNs problem that allows the extension of natural gradient to it, with proved connections to Green's function theory.