A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems

TL;DR

This paper introduces PINTO, a physics-informed transformer neural operator that efficiently generalizes to new initial and boundary conditions for PDEs without retraining, using physics loss and innovative attention-based integral operators.

Contribution

The paper proposes a novel transformer-based neural operator architecture with iterative kernel units that enable physics-informed learning without simulation data, improving generalization to unseen boundary conditions.

Findings

PINTO achieves significantly lower errors than existing methods on benchmark PDEs.

It generalizes well to unseen initial and boundary conditions without retraining.

Accurately predicts solutions at untrained time steps for advection and Burgers equations.

Abstract

Initial boundary value problems arise commonly in applications with engineering and natural systems governed by nonlinear partial differential equations (PDEs). Operator learning is an emerging field for solving these equations by using a neural network to learn a map between infinite dimensional input and output function spaces. These neural operators are trained using a combination of data (observations or simulations) and PDE-residuals (physics-loss). A major drawback of existing neural approaches is the requirement to retrain with new initial/boundary conditions, and the necessity for a large amount of simulation data for training. We develop a physics-informed transformer neural operator (named PINTO) that efficiently generalizes to unseen initial and boundary conditions, trained in a simulation-free setting using only physics loss. The main innovation lies in our new iterative kernel integral operator units, implemented using cross-attention, to transform the PDE solution's domain points into an initial/boundary condition-aware representation vector, enabling efficient learning of the solution function for new scenarios. The PINTO architecture is applied to simulate the solutions of important equations used in engineering applications: advection, Burgers, and steady and unsteady Navier-Stokes equations (three flow scenarios). For these five test cases, we show that the relative errors during testing under challenging conditions of unseen initial/boundary conditions are only one-fifth to one-third of other leading physics informed operator learning methods. Moreover, our PINTO model is able to accurately solve the advection and Burgers equations at time steps that are not included in the training collocation points. The code is available at https://github.com/quest-lab-iisc/PINTO