Efficient Generative Transformer Operators For Million-Point PDEs

TL;DR

ECHO is a novel transformer-based framework that efficiently generates high-resolution PDE trajectories on dense grids, overcoming scalability and error accumulation issues of previous neural operators.

Contribution

The paper introduces ECHO, a hierarchical convolutional transformer architecture with a new training strategy for scalable, high-fidelity PDE trajectory generation from sparse inputs.

Findings

Achieves 100x spatio-temporal compression while maintaining fidelity.

Enables high-resolution PDE solutions from sparse input data.

Demonstrates state-of-the-art results on complex, long-horizon PDE simulations.

Abstract

We introduce ECHO, a transformer-operator framework for generating million-point PDE trajectories. While existing neural operators (NOs) have shown promise for solving partial differential equations, they remain limited in practice due to poor scalability on dense grids, error accumulation during dynamic unrolling, and task-specific design. ECHO addresses these challenges through three key innovations. (i) It employs a hierarchical convolutional encode-decode architecture that achieves a 100 $\times$ spatio-temporal compression while preserving fidelity on mesh points. (ii) It incorporates a training and adaptation strategy that enables high-resolution PDE solution generation from sparse input grids. (iii) It adopts a generative modeling paradigm that learns complete trajectory segments, mitigating long-horizon error drift. The training strategy decouples representation learning from downstream task supervision, allowing the model to tackle multiple tasks such as trajectory generation, forward and inverse problems, and interpolation. The generative model further supports both conditional and unconditional generation. We demonstrate state-of-the-art performance on million-point simulations across diverse PDE systems featuring complex geometries, high-frequency dynamics, and long-term horizons.