Towards a Physics Foundation Model

TL;DR

This paper introduces GPhyT, a transformer-based physics model trained on diverse data that can simulate multiple physical phenomena, generalize to unseen systems, and outperform specialized models, paving the way for a universal Physics Foundation Model.

Contribution

The paper presents GPhyT, the first large-scale transformer model capable of learning and generalizing physical dynamics across multiple domains without retraining.

Findings

GPhyT outperforms specialized models by over 7x in multiple physics tasks.

GPhyT demonstrates zero-shot generalization to unseen physical systems.

GPhyT provides more stable long-term predictions through long-horizon rollouts.

Abstract

Foundation models have revolutionized natural language processing through a ``train once, deploy anywhere'' paradigm, where a single pre-trained model adapts to countless downstream tasks without retraining. Access to a Physics Foundation Model (PFM) would be transformative - democratizing access to high-fidelity simulations, accelerating scientific discovery, and eliminating the need for specialized solver development. Yet current physics-aware machine learning approaches remain fundamentally limited to single, narrow domains and require retraining for each new system. We present the General Physics Transformer (GPhyT), trained on 1.8 TB of diverse simulation data, that demonstrates foundation model capabilities are achievable for physics. Our key insight is that transformers can learn to infer governing dynamics from context, enabling a single model to simulate fluid-solid interactions, shock waves, thermal convection, and multi-phase dynamics without being told the underlying equations. GPhyT achieves three critical breakthroughs: (1) superior performance across multiple physics domains, outperforming specialized architectures by more than 7x, (2) plausible zero-shot generalization to entirely unseen physical systems through in-context learning, and (3) more stable long-term predictions through long-horizon rollouts. By establishing that a single model can learn generalizable physical principles from data alone, this work opens the path toward a universal PFM that could transform computational science and engineering.