Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising

TL;DR

X-WAM is a unified 4D world model that combines real-time robotic action and high-fidelity 4D world synthesis using video priors, with an asynchronous denoising method for efficient inference.

Contribution

It introduces X-WAM, a novel framework integrating 4D world modeling and action execution, leveraging pretrained video diffusion models and asynchronous noise sampling for efficiency.

Findings

Achieves 79.2% success on RoboCasa benchmark

Achieves 90.7% success on RoboTwin 2.0 benchmark

Produces superior 4D reconstruction and generation results

Abstract

We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruction) in a single framework, addressing the critical limitations of prior unified world models (e.g., UWM) that only model 2D pixel-space and fail to balance action efficiency and world modeling quality. To leverage the strong visual priors of pretrained video diffusion models, X-WAM imagines the future world by predicting multi-view RGB-D videos, and obtains spatial information efficiently through a lightweight structural adaptation: replicating the final few blocks of the pretrained Diffusion Transformer into a dedicated depth prediction branch for the reconstruction of future spatial information. Moreover, we propose Asynchronous Noise Sampling (ANS) to jointly optimize generation quality and action decoding efficiency. ANS applies a specialized asynchronous denoising schedule during inference, which rapidly decodes actions with fewer steps to enable efficient real-time execution, while dedicating the full sequence of steps to generate high-fidelity video. Rather than entirely decoupling the timesteps during training, ANS samples from their joint distribution to align with the inference distribution. Pretrained on over 5,800 hours of robotic data, X-WAM achieves 79.2% and 90.7% average success rate on RoboCasa and RoboTwin 2.0 benchmarks, while producing high-fidelity 4D reconstruction and generation surpassing existing methods in both visual and geometric metrics.