Solving Differential Equations with Constrained Learning

TL;DR

This paper introduces a science-constrained learning framework for solving PDEs that improves accuracy and robustness by incorporating structural constraints and prior knowledge, reducing hyperparameter sensitivity and computational costs.

Contribution

It develops a novel constrained learning approach that addresses limitations of existing neural PDE solvers, enabling more reliable and efficient solutions with less hyperparameter tuning.

Findings

The SCL framework achieves accurate PDE solutions across various architectures.

It naturally incorporates structural constraints and prior knowledge.

The method often reduces computational costs compared to traditional neural approaches.

Abstract

(Partial) differential equations (PDEs) are fundamental tools for describing natural phenomena, making their solution crucial in science and engineering. While traditional methods, such as the finite element method, provide reliable solutions, their accuracy is often tied to the use of computationally intensive fine meshes. Moreover, they do not naturally account for measurements or prior solutions, and any change in the problem parameters requires results to be fully recomputed. Neural network-based approaches, such as physics-informed neural networks and neural operators, offer a mesh-free alternative by directly fitting those models to the PDE solution. They can also integrate prior knowledge and tackle entire families of PDEs by simply aggregating additional training losses. Nevertheless, they are highly sensitive to hyperparameters such as collocation points and the weights associated with each loss. This paper addresses these challenges by developing a science-constrained learning (SCL) framework. It demonstrates that finding a (weak) solution of a PDE is equivalent to solving a constrained learning problem with worst-case losses. This explains the limitations of previous methods that minimize the expected value of aggregated losses. SCL also organically integrates structural constraints (e.g., invariances) and (partial) measurements or known solutions. The resulting constrained learning problems can be tackled using a practical algorithm that yields accurate solutions across a variety of PDEs, neural network architectures, and prior knowledge levels without extensive hyperparameter tuning and sometimes even at a lower computational cost.