Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks

TL;DR

This paper introduces a theoretical framework for Physically Informed Neural Networks (PINNs), revealing how architecture choices influence training bias through spectral analysis, and connects neural networks with Gaussian process regression for better understanding.

Contribution

It develops an integro-differential equation linking PINN predictions to kernel choices, providing insights into implicit bias and spectral properties of PINNs.

Findings

Derives a neurally-informed integro-differential equation for PINNs.

Connects neural networks with Gaussian process regression.

Quantifies implicit bias via spectral decomposition.

Abstract

Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design and training protocol requires careful craftsmanship. Here, we suggest a comprehensive theoretical framework that sheds light on this important problem. Leveraging an equivalence between infinitely over-parameterized neural networks and Gaussian process regression (GPR), we derive an integro-differential equation that governs PINN prediction in the large data-set limit -- the neurally-informed equation. This equation augments the original one by a kernel term reflecting architecture choices and allows quantifying implicit bias induced by the network via a spectral decomposition of the source term in the original differential equation.