INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling

TL;DR

INSPATIO-WORLD is a real-time 4D world simulator that uses a novel spatiotemporal autoregressive model to generate high-fidelity, interactive scenes from a single video, improving spatial consistency and realism.

Contribution

The paper introduces INSPATIO-WORLD, a framework with a STAR architecture and JDMD training method, enabling realistic, controllable, and consistent 4D scene generation from minimal input.

Findings

Outperforms state-of-the-art models in spatial consistency and interaction accuracy.

Ranks first on the WorldScore-Dynamic benchmark among real-time interactive methods.

Effectively uses real-world data distribution regularization to enhance fidelity.

Abstract

Building world models with spatial consistency and real-time interactivity remains a fundamental challenge in computer vision. Current video generation paradigms often struggle with a lack of spatial persistence and insufficient visual realism, making it difficult to support seamless navigation in complex environments. To address these challenges, we propose INSPATIO-WORLD, a novel real-time framework capable of recovering and generating high-fidelity, dynamic interactive scenes from a single reference video. At the core of our approach is a Spatiotemporal Autoregressive (STAR) architecture, which enables consistent and controllable scene evolution through two tightly coupled components: Implicit Spatiotemporal Cache aggregates reference and historical observations into a latent world representation, ensuring global consistency during long-horizon navigation; Explicit Spatial Constraint Module enforces geometric structure and translates user interactions into precise and physically plausible camera trajectories. Furthermore, we introduce Joint Distribution Matching Distillation (JDMD). By using real-world data distributions as a regularizing guide, JDMD effectively overcomes the fidelity degradation typically caused by over-reliance on synthetic data. Extensive experiments demonstrate that INSPATIO-WORLD significantly outperforms existing state-of-the-art (SOTA) models in spatial consistency and interaction precision, ranking first among real-time interactive methods on the WorldScore-Dynamic benchmark, and establishing a practical pipeline for navigating 4D environments reconstructed from monocular videos.