RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data

TL;DR

RealPDEBench introduces a comprehensive benchmark integrating real-world measurements with simulations for complex physical systems, enabling better evaluation and development of scientific ML models to bridge the sim-to-real gap.

Contribution

This work presents the first benchmark combining real-world and simulated data for physical systems, including datasets, tasks, metrics, and baselines to advance scientific ML research.

Findings

Pretraining with simulated data improves accuracy and convergence.

Significant discrepancies exist between real-world and simulated data.

Benchmark facilitates comparison of models on real-world physical data.

Abstract

Predicting the evolution of complex physical systems remains a central problem in science and engineering. Despite rapid progress in scientific Machine Learning (ML) models, a critical bottleneck is the lack of expensive real-world data, resulting in most current models being trained and validated on simulated data. Beyond limiting the development and evaluation of scientific ML, this gap also hinders research into essential tasks such as sim-to-real transfer. We introduce RealPDEBench, the first benchmark for scientific ML that integrates real-world measurements with paired numerical simulations. RealPDEBench consists of five datasets, three tasks, eight metrics, and ten baselines. We first present five real-world measured datasets with paired simulated datasets across different complex physical systems. We further define three tasks, which allow comparisons between real-world and simulated data, and facilitate the development of methods to bridge the two. Moreover, we design eight evaluation metrics, spanning data-oriented and physics-oriented metrics, and finally benchmark ten representative baselines, including state-of-the-art models, pretrained PDE foundation models, and a traditional method. Experiments reveal significant discrepancies between simulated and real-world data, while showing that pretraining with simulated data consistently improves both accuracy and convergence. In this work, we hope to provide insights from real-world data, advancing scientific ML toward bridging the sim-to-real gap and real-world deployment. Our benchmark, datasets, and instructions are available at https://realpdebench.github.io/.