A Simple but Efficient Transformer-Based Physics-Informed Neural Network for Incompressible Navier--Stokes Equations

TL;DR

PhysicsFormer is a Transformer-based physics-informed neural network that efficiently simulates complex fluid flows, offering faster training, lower computational cost, and high accuracy in both forward and inverse problems.

Contribution

It introduces a lightweight Transformer architecture with pseudo-sequential spatio-temporal representations and a dynamics-weighted loss for improved fluid flow modeling.

Findings

Achieves faster training and lower computational cost than existing Transformer PINNs.

Accurately reconstructs flow fields and identifies parameters with nearly 0% error.

Effectively models high-Reynolds-number flows with limited spatial measurements.

Abstract

Traditional computational fluid dynamics and physics-informed neural networks (PINNs) often suffer from high computational cost, mesh sensitivity, and reduced accuracy for strongly nonlinear and time-dependent flows. To address these limitations, we propose \textit{PhysicsFormer}, a simple and efficient Transformer-based physics-informed neural network framework for complex fluid flow simulations. The proposed architecture employs encoder--decoder multi-head attention to capture long-range temporal dependencies and enhance spatio-temporal information propagation. Unlike conventional multilayer perceptron-based PINNs, \textit{PhysicsFormer} utilizes pseudo-sequential spatio-temporal representations together with a dynamics-weighted loss formulation to improve convergence, stability, and predictive accuracy. Owing to its lightweight architecture and parallel learning strategy, the proposed framework achieves faster training and lower computational cost than existing Transformer-based PINN models. The performance of the proposed framework is demonstrated on the convection equation, Burgers' equation, lid-driven cavity flow at $Re=100$, and inverse Navier--Stokes and flow reconstruction problems for flow past a circular cylinder at $Re=100$ and $Re=3900$. For the inverse Navier--Stokes problem at $Re=100$, the proposed framework simultaneously reconstructs the flow field and identifies governing equation parameters with nearly $0\%$ absolute error under both clean and noisy data conditions. Furthermore, for the high-Reynolds-number case at $Re=3900$, \textit{PhysicsFormer} accurately reconstructs the velocity and pressure fields using only $25$ spatial measurements per snapshot over $100$ temporal snapshots. The obtained results demonstrate that \textit{PhysicsFormer} provides an accurate, robust, and computationally efficient framework for complex time-dependent fluid flow problems.