An Expert's Guide to Training Physics-informed Neural Networks

TL;DR

This paper offers best practices, benchmark problems, and an optimized JAX library to improve the training efficiency and accuracy of physics-informed neural networks, addressing common challenges and guiding future research.

Contribution

It introduces a set of best practices, challenging benchmarks, and a comprehensive ablation study for PINNs, along with a highly optimized JAX library for reproducibility and future work.

Findings

Enhanced training efficiency and accuracy of PINNs.

Identification of architecture and training strategies affecting performance.

State-of-the-art results achieved with proposed methods.

Abstract

Physics-informed neural networks (PINNs) have been popularized as a deep learning framework that can seamlessly synthesize observational data and partial differential equation (PDE) constraints. Their practical effectiveness however can be hampered by training pathologies, but also oftentimes by poor choices made by users who lack deep learning expertise. In this paper we present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs. We also put forth a series of challenging benchmark problems that highlight some of the most prominent difficulties in training PINNs, and present comprehensive and fully reproducible ablation studies that demonstrate how different architecture choices and training strategies affect the test accuracy of the resulting models. We show that the methods and guiding principles put forth in this study lead to state-of-the-art results and provide strong baselines that future studies should use for comparison purposes. To this end, we also release a highly optimized library in JAX that can be used to reproduce all results reported in this paper, enable future research studies, as well as facilitate easy adaptation to new use-case scenarios.