Sobolev Training for Physics Informed Neural Networks

TL;DR

This paper introduces Sobolev-PINNs, a new loss function for physics-informed neural networks that significantly improves training efficiency and convergence speed by incorporating derivative information and Sobolev space error bounds.

Contribution

The paper proposes Sobolev-PINNs, a novel loss function that enhances PINNs training by leveraging Sobolev space properties, leading to faster convergence and better high-dimensional PDE solutions.

Findings

Sobolev-PINNs achieve faster convergence than traditional $L^2$ loss PINNs.

Theoretical bounds show error reduction in Sobolev spaces for specific PDEs.

Empirical results demonstrate improved performance in high-dimensional PDEs.

Abstract

Physics Informed Neural Networks (PINNs) is a promising application of deep learning. The smooth architecture of a fully connected neural network is appropriate for finding the solutions of PDEs; the corresponding loss function can also be intuitively designed and guarantees the convergence for various kinds of PDEs. However, the rate of convergence has been considered as a weakness of this approach. This paper proposes Sobolev-PINNs, a novel loss function for the training of PINNs, making the training substantially efficient. Inspired by the recent studies that incorporate derivative information for the training of neural networks, we develop a loss function that guides a neural network to reduce the error in the corresponding Sobolev space. Surprisingly, a simple modification of the loss function can make the training process similar to \textit{Sobolev Training} although PINNs is not a fully supervised learning task. We provide several theoretical justifications that the proposed loss functions upper bound the error in the corresponding Sobolev spaces for the viscous Burgers equation and the kinetic Fokker--Planck equation. We also present several simulation results, which show that compared with the traditional $L^2$ loss function, the proposed loss function guides the neural network to a significantly faster convergence. Moreover, we provide the empirical evidence that shows that the proposed loss function, together with the iterative sampling techniques, performs better in solving high dimensional PDEs.