Learning differentiable solvers for systems with hard constraints

TL;DR

This paper presents a neural network approach that incorporates differentiable PDE constraints directly into the architecture, enabling accurate and physically consistent solutions for PDE problems.

Contribution

The authors develop a differentiable PDE-constrained layer integrated into neural networks, allowing enforcement of physical constraints via optimization and the implicit function theorem.

Findings

Significantly lower test error with constrained models

Continuous solutions that satisfy physical constraints

Effective enforcement of PDE constraints in neural networks

Abstract

We introduce a practical method to enforce partial differential equation (PDE) constraints for functions defined by neural networks (NNs), with a high degree of accuracy and up to a desired tolerance. We develop a differentiable PDE-constrained layer that can be incorporated into any NN architecture. Our method leverages differentiable optimization and the implicit function theorem to effectively enforce physical constraints. Inspired by dictionary learning, our model learns a family of functions, each of which defines a mapping from PDE parameters to PDE solutions. At inference time, the model finds an optimal linear combination of the functions in the learned family by solving a PDE-constrained optimization problem. Our method provides continuous solutions over the domain of interest that accurately satisfy desired physical constraints. Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective.