The Fourier Spectral Transformer Networks For Efficient and Generalizable Nonlinear PDEs Prediction

TL;DR

This paper introduces a Fourier Spectral Transformer network that combines spectral methods and attention mechanisms to accurately predict nonlinear PDEs, demonstrating superior long-term forecasting and generalization capabilities.

Contribution

It presents a novel spectral Transformer architecture that effectively models spectral coefficient evolution for nonlinear PDEs, outperforming traditional methods.

Findings

Achieves high accuracy in long-term predictions of PDEs.

Outperforms traditional numerical and machine learning methods.

Generalizes well to unseen data.

Abstract

In this work we propose a unified Fourier Spectral Transformer network that integrates the strengths of classical spectral methods and attention based neural architectures. By transforming the original PDEs into spectral ordinary differential equations, we use high precision numerical solvers to generate training data and use a Transformer network to model the evolution of the spectral coefficients. We demonstrate the effectiveness of our approach on the two dimensional incompressible Navier-Stokes equations and the one dimensional Burgers' equation. The results show that our spectral Transformer can achieve highly accurate long term predictions even with limited training data, better than traditional numerical methods and machine learning methods in forecasting future flow dynamics. The proposed framework generalizes well to unseen data, bringing a promising paradigm for real time prediction and control of complex dynamical systems.