EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale

TL;DR

EddyFormer is a Transformer-based spectral-element neural network that accelerates large-scale 3D turbulence simulations, achieving DNS-level accuracy with significant speedup and good domain generalization.

Contribution

The paper introduces EddyFormer, a novel spectral-element Transformer architecture that combines spectral methods with attention mechanisms for efficient, accurate turbulence simulation at scale.

Findings

Achieves DNS-level accuracy at 256^3 resolution with 30x speedup.

Successfully generalizes to larger domains up to 4x in size.

Outperforms prior ML models on the Well turbulence benchmark.

Abstract

Computationally resolving turbulence remains a central challenge in fluid dynamics due to its multi-scale interactions. Fully resolving large-scale turbulence through direct numerical simulation (DNS) is computationally prohibitive, motivating data-driven machine learning alternatives. In this work, we propose EddyFormer, a Transformer-based spectral-element (SEM) architecture for large-scale turbulence simulation that combines the accuracy of spectral methods with the scalability of the attention mechanism. We introduce an SEM tokenization that decomposes the flow into grid-scale and subgrid-scale components, enabling capture of both local and global features. We create a new three-dimensional isotropic turbulence dataset and train EddyFormer to achieves DNS-level accuracy at 256^3 resolution, providing a 30x speedup over DNS. When applied to unseen domains up to 4x larger than in training, EddyFormer preserves accuracy on physics-invariant metrics-energy spectra, correlation functions, and structure functions-showing domain generalization. On The Well benchmark suite of diverse turbulent flows, EddyFormer resolves cases where prior ML models fail to converge, accurately reproducing complex dynamics across a wide range of physical conditions.