Spatia: Video Generation with Updatable Spatial Memory

TL;DR

Spatia introduces a novel video generation framework that uses a persistent 3D scene memory and visual SLAM to improve spatial consistency and enable interactive editing, addressing limitations of existing models.

Contribution

It proposes a spatial memory-aware framework that explicitly maintains a 3D scene point cloud for scalable, consistent, and editable video generation.

Findings

Enhanced spatial consistency in generated videos.

Ability to perform explicit camera control and 3D-aware editing.

Effective integration of visual SLAM for memory updates.

Abstract

Existing video generation models struggle to maintain long-term spatial and temporal consistency due to the dense, high-dimensional nature of video signals. To overcome this limitation, we propose Spatia, a spatial memory-aware video generation framework that explicitly preserves a 3D scene point cloud as persistent spatial memory. Spatia iteratively generates video clips conditioned on this spatial memory and continuously updates it through visual SLAM. This dynamic-static disentanglement design enhances spatial consistency throughout the generation process while preserving the model's ability to produce realistic dynamic entities. Furthermore, Spatia enables applications such as explicit camera control and 3D-aware interactive editing, providing a geometrically grounded framework for scalable, memory-driven video generation.