TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model

TL;DR

TeleWorld introduces a real-time 4D multimodal world model that unifies video generation, scene reconstruction, and memory, enabling coherent, long-term dynamic scene synthesis with practical computational efficiency.

Contribution

It presents a novel closed-loop framework combining generation, reconstruction, and guidance, with hierarchical planning and distillation techniques for real-time, long-horizon multimodal scene modeling.

Findings

Achieves high-quality, consistent dynamic scene generation in real-time.

Effectively integrates static and dynamic scene components within a unified 4D model.

Demonstrates strong performance in long-term scene understanding and multimodal synthesis.

Abstract

World models aim to endow AI systems with the ability to represent, generate, and interact with dynamic environments in a coherent and temporally consistent manner. While recent video generation models have demonstrated impressive visual quality, they remain limited in real-time interaction, long-horizon consistency, and persistent memory of dynamic scenes, hindering their evolution into practical world models. In this report, we present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system. TeleWorld introduces a novel generation-reconstruction-guidance paradigm, where generated video streams are continuously reconstructed into a dynamic 4D spatio-temporal representation, which in turn guides subsequent generation to maintain spatial, temporal, and physical consistency. To support long-horizon generation with low latency, we employ an autoregressive diffusion-based video model enhanced with Macro-from-Micro Planning (MMPL)--a hierarchical planning method that reduces error accumulation from frame-level to segment-level-alongside efficient Distribution Matching Distillation (DMD), enabling real-time synthesis under practical computational budgets. Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible systems. Extensive experiments demonstrate that TeleWorld achieves strong performance in both static and dynamic world understanding, long-term consistency, and real-time generation efficiency, positioning it as a practical step toward interactive, memory-enabled world models for multimodal generation and embodied intelligence.