QP Based Constrained Optimization for Reliable PINN Training

TL;DR

This paper introduces a quadratic programming-based training scheme for Physics-Informed Neural Networks (PINNs) that improves convergence, stability, and loss function design by framing training as a constrained optimization problem.

Contribution

The paper presents a novel QP-based gradient descent method for PINNs, enabling dynamic loss balancing and guaranteed convergence to optimal parameters.

Findings

Effective training of PINNs with noisy data

Improved convergence and stability in PINN training

Successful application to Laplace's equation in complex geometries

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for integrating physics-based constraints and data to address forward and inverse problems in machine learning. Despite their potential, the implementation of PINNs are hampered by several challenges, including issues related to convergence, stability, and the design of neural networks and loss functions. In this paper, we introduce a novel training scheme that addresses these challenges by framing the training process as a constrained optimization problem. Utilizing a quadratic program (QP)-based gradient descent law, our approach simplifies the design of loss functions and guarantees convergences to optimal neural network parameters. This methodology enables dynamic balancing, over the course of training, between data-based loss and a partial differential equation (PDE) residual loss, ensuring an acceptable level of accuracy while prioritizing the minimization of PDE-based loss. We demonstrate the formulation of the constrained PINNs approach with noisy data, in the context of solving Laplace's equation in a capacitor with complex geometry. This work not only advances the capabilities of PINNs but also provides a framework for their training.