IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Video

TL;DR

IRIS introduces a comprehensive real-world benchmark dataset with standardized evaluation protocols for inverse recovery and identification of physical dynamic systems from monocular video, addressing gaps in existing synthetic and limited real-world datasets.

Contribution

This work provides the first large-scale, high-fidelity real-world dataset with ground-truth parameters and governing equations for physical system identification from video, along with a standardized evaluation protocol.

Findings

Multiple baseline methods evaluated, revealing systematic failure modes.

Benchmark enables principled comparison of parameter estimation and equation identification.

Dataset and tools publicly released for future research.

Abstract

Unsupervised physical parameter estimation from video lacks a common benchmark: existing methods evaluate on non-overlapping synthetic data, the sole real-world dataset is restricted to single-body systems, and no established protocol addresses governing-equation identification. This work introduces IRIS, a high-fidelity benchmark comprising 220 real-world videos captured at 4K resolution and 60\,fps, spanning both single- and multi-body dynamics with independently measured ground-truth parameters and uncertainty estimates. Each dynamical system is recorded under controlled laboratory conditions and paired with its governing equations, enabling principled evaluation. A standardized evaluation protocol is defined encompassing parameter accuracy, identifiability, extrapolation, robustness, and governing-equation selection. Multiple baselines are evaluated, including a multi-step physics loss formulation and four complementary equation-identification strategies (VLM temporal reasoning, describe-then-classify prompting, CNN-based classification, and path-based labelling), establishing reference performance across all IRIS scenarios and exposing systematic failure modes that motivate future research. The dataset, annotations, evaluation toolkit, and all baseline implementations are publicly released.