Towards a Foundation Model for Partial Differential Equations Across Physics Domains

TL;DR

PDE-FM is a versatile foundation model for physics-informed machine learning that unifies reasoning across diverse PDE systems, achieving state-of-the-art accuracy and strong generalization in multiple physics domains.

Contribution

It introduces PDE-FM, a pretrained, modular foundation model that unifies spatial, spectral, and temporal reasoning for PDEs across physics domains, enabling transferability without retraining.

Findings

Achieves 46% reduction in VRMSE on diverse PDE datasets.

Demonstrates robust cross-physics generalization, especially in turbulent and radiative systems.

Outperforms prior neural operator baselines in accuracy.

Abstract

We present PDE-FM, a modular foundation model for physics-informed machine learning that unifies spatial, spectral, and temporal reasoning across heterogeneous partial differential equation (PDE) systems. PDE-FM combines spatial-spectral tokenization, physics-aware conditioning, and a Mamba-based state-space backbone with an operator-theoretic decoder, enabling scalable and data-efficient modeling of complex physical dynamics. In contrast to task-specific neural operators, PDE-FM is pretrained once on diverse PDE datasets and can be transferred to new physical regimes without architectural or data-specific modifications. Evaluated on twelve 2D and 3D datasets from The Well benchmark - spanning hydrodynamic, radiative, elastic, and astrophysical phenomena - PDE-FM achieves state-of-the-art accuracy in six domains, reducing mean VRMSE by 46% relative to prior operator-learning baselines. The model demonstrates robust cross-physics generalization, excelling in turbulent and radiative systems while maintaining strong performance in linear and steady-state regimes. These results suggest that large-scale pretraining across diverse physical processes can yield transferable representations of dynamics, marking a step toward unified, foundation-level surrogates for multi-physics simulation and scientific discovery.