Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems

TL;DR

This paper introduces several innovations to the PECANN framework, including a generalized ALM, a constraint aggregation technique, Fourier feature mapping, time-windowing, and a novel adaptive penalty update strategy, significantly enhancing its ability to solve complex PDEs efficiently.

Contribution

The paper's main contribution is the development of a conditionally adaptive penalty update (CAPU) strategy for ALM within PECANN, improving constraint enforcement and training efficiency for challenging PDE problems.

Findings

PECANN-CAPU achieves high accuracy on diverse PDE problems.

The adaptive penalty strategy accelerates constraint enforcement.

The framework outperforms or matches recent methods in scientific computing tasks.

Abstract

We present several key advances to the Physics and Equality Constrained Artificial Neural Networks (PECANN) framework, substantially improving its capacity to solve challenging partial differential equations (PDEs). Our enhancements broaden the framework's applicability and improve efficiency. First, we generalize the Augmented Lagrangian Method (ALM) to support multiple, independent penalty parameters for enforcing heterogeneous constraints. Second, we introduce a constraint aggregation technique to address inefficiencies associated with point-wise enforcement. Third, we incorporate a single Fourier feature mapping to capture highly oscillatory solutions with multi-scale features, where alternative methods often require multiple mappings or costlier architectures. Fourth, a novel time-windowing strategy enables seamless long-time evolution without relying on discrete time models. Fifth, and critically, we propose a conditionally adaptive penalty update (CAPU) strategy for ALM that accelerates the growth of Lagrange multipliers for constraints with larger violations, while enabling coordinated updates of multiple penalty parameters. CAPU accelerates the growth of Lagrange multipliers for selectively challenging constraints, enhancing constraint enforcement during training. We demonstrate the effectiveness of PECANN-CAPU across diverse problems, including the transonic rarefaction problem, reversible scalar advection by a vortex, high-wavenumber Helmholtz and Poisson's equations, and inverse heat source identification. The framework achieves competitive accuracy across all cases when compared with established methods and recent approaches based on Kolmogorov-Arnold networks. Collectively, these advances improve the robustness, computational efficiency, and applicability of PECANN to demanding problems in scientific computing.