Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit

TL;DR

This paper introduces a scalable multiscale transformer model for turbulence simulation that captures small-scale eddies at exascale, achieving high efficiency on supercomputers and enabling detailed turbulence modeling.

Contribution

The paper presents a novel hierarchical Turbulence Transformer with RingX parallelism, enabling long-context turbulence modeling at exascale with high efficiency.

Findings

Achieves 1.1 EFLOPS performance on Frontier supercomputer.

Maintains 94% scaling efficiency across 32,768 GPUs.

First AI model capable of resolving small-scale turbulence down to the viscous range.

Abstract

Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range.