Physics informed neural networks for elliptic equations with oscillatory differential operators

TL;DR

This paper investigates the challenges of using physics-informed neural networks (PINNs) to solve elliptic PDEs with oscillatory coefficients, revealing that multiscale features cause training difficulties due to the growth of the neural tangent kernel.

Contribution

The study explains why standard PINNs struggle with multiscale elliptic equations, linking the difficulty to the growth of the neural tangent kernel with decreasing scale parameter.

Findings

NTK norm grows as 1/ε^2 for oscillatory coefficients

Training difficulty increases with scale separation

Numerical examples demonstrate optimization stiffness

Abstract

Physics informed neural network (PINN) based solution methods for differential equations have recently shown success in a variety of scientific computing applications. Several authors have reported difficulties, however, when using PINNs to solve equations with multiscale features. The objective of the present work is to illustrate and explain the difficulty of using standard PINNs for the particular case of divergence-form elliptic partial differential equations (PDEs) with oscillatory coefficients present in the differential operator. We show that if the coefficient in the elliptic operator $a^{\epsilon}(x)$ is of the form $a(x/\epsilon)$ for a 1-periodic coercive function $a(\cdot)$, then the Frobenius norm of the neural tangent kernel (NTK) matrix associated to the loss function grows as $1/\epsilon^2$. This implies that as the separation of scales in the problem increases, training the neural network with gradient descent based methods to achieve an accurate approximation of the solution to the PDE becomes increasingly difficult. Numerical examples illustrate the stiffness of the optimization problem.