Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory

TL;DR

Infinite-World introduces a hierarchical, pose-free memory system that enables interactive world models to maintain coherent visual memory over 1000+ frames in complex environments, overcoming challenges of noisy pose estimations and viewpoint revisits.

Contribution

The paper presents a novel Hierarchical Pose-free Memory Compressor and an Uncertainty-aware Action Labeling module, advancing long-range visual memory and robust action learning in real-world videos.

Findings

Achieves over 1000-frame coherent visual memory in real-world environments.

Outperforms existing models in visual quality and action controllability.

Demonstrates effective long-range loop-closure with minimal fine-tuning.

Abstract

We propose Infinite-World, a robust interactive world model capable of maintaining coherent visual memory over 1000+ frames in complex real-world environments. While existing world models can be efficiently optimized on synthetic data with perfect ground-truth, they lack an effective training paradigm for real-world videos due to noisy pose estimations and the scarcity of viewpoint revisits. To bridge this gap, we first introduce a Hierarchical Pose-free Memory Compressor (HPMC) that recursively distills historical latents into a fixed-budget representation. By jointly optimizing the compressor with the generative backbone, HPMC enables the model to autonomously anchor generations in the distant past with bounded computational cost, eliminating the need for explicit geometric priors. Second, we propose an Uncertainty-aware Action Labeling module that discretizes continuous motion into a tri-state logic. This strategy maximizes the utilization of raw video data while shielding the deterministic action space from being corrupted by noisy trajectories, ensuring robust action-response learning. Furthermore, guided by insights from a pilot toy study, we employ a Revisit-Dense Finetuning Strategy using a compact, 30-minute dataset to efficiently activate the model's long-range loop-closure capabilities. Extensive experiments, including objective metrics and user studies, demonstrate that Infinite-World achieves superior performance in visual quality, action controllability, and spatial consistency.