A connection between probability, physics and neural networks

TL;DR

This paper proposes a method to design neural networks that inherently obey physical laws by leveraging Gaussian process theory and the infinite-width limit, ensuring physics compliance through kernel constraints.

Contribution

It introduces a framework linking neural networks, Gaussian processes, and physical laws, enabling the construction of physics-informed neural networks via kernel and activation function design.

Findings

Neural networks can be constrained to obey physical laws through kernel design.

The approach uses the infinite-width limit and Gaussian process theory to incorporate physics.

Examples with the 1D-Helmholtz equation demonstrate the method's effectiveness.

Abstract

We illustrate an approach that can be exploited for constructing neural networks which a priori obey physical laws. We start with a simple single-layer neural network (NN) but refrain from choosing the activation functions yet. Under certain conditions and in the infinite-width limit, we may apply the central limit theorem, upon which the NN output becomes Gaussian. We may then investigate and manipulate the limit network by falling back on Gaussian process (GP) theory. It is observed that linear operators acting upon a GP again yield a GP. This also holds true for differential operators defining differential equations and describing physical laws. If we demand the GP, or equivalently the limit network, to obey the physical law, then this yields an equation for the covariance function or kernel of the GP, whose solution equivalently constrains the model to obey the physical law. The central limit theorem then suggests that NNs can be constructed to obey a physical law by choosing the activation functions such that they match a particular kernel in the infinite-width limit. The activation functions constructed in this way guarantee the NN to a priori obey the physics, up to the approximation error of non-infinite network width. Simple examples of the homogeneous 1D-Helmholtz equation are discussed and compared to naive kernels and activations.