Geometry-Aware Rotary Position Embedding for Consistent Video World Model

TL;DR

This paper introduces ViewRope, a geometry-aware encoding for video transformers that enhances 3D consistency and long-term stability in predictive world models by integrating camera-ray directions into attention mechanisms.

Contribution

The paper proposes ViewRope, a novel geometry-aware positional encoding that improves 3D consistency in video transformers for world modeling, addressing limitations of screen-space embeddings.

Findings

Significantly improves long-term scene consistency.

Reduces computational costs while maintaining memory fidelity.

Enhances loop-closure accuracy in predictive models.

Abstract

Predictive world models that simulate future observations under explicit camera control are fundamental to interactive AI. Despite rapid advances, current systems lack spatial persistence: they fail to maintain stable scene structures over long trajectories, frequently hallucinating details when cameras revisit previously observed locations. We identify that this geometric drift stems from reliance on screen-space positional embeddings, which conflict with the projective geometry required for 3D consistency. We introduce \textbf{ViewRope}, a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers. By parameterizing attention with relative ray geometry rather than pixel locality, ViewRope provides a model-native inductive bias for retrieving 3D-consistent content across temporal gaps. We further propose \textbf{Geometry-Aware Frame-Sparse Attention}, which exploits these geometric cues to selectively attend to relevant historical frames, improving efficiency without sacrificing memory consistency. We also present \textbf{ViewBench}, a diagnostic suite measuring loop-closure fidelity and geometric drift. Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.