WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation

TL;DR

WorldWander introduces a novel framework for translating videos between egocentric and exocentric perspectives, leveraging advanced diffusion transformers and a new dataset to improve synchronization and consistency.

Contribution

The paper presents WorldWander, a new in-context learning approach with specialized modules and a large-scale dataset for cross-view video translation.

Findings

Achieves superior perspective synchronization

Maintains character consistency across views

Sets a new benchmark in egocentric-exocentric video translation

Abstract

Video diffusion models have recently achieved remarkable progress in realism and controllability. However, achieving seamless video translation across different perspectives, such as first-person (egocentric) and third-person (exocentric), remains underexplored. Bridging these perspectives is crucial for filmmaking, embodied AI, and world models. Motivated by this, we present WorldWander, an in-context learning framework tailored for translating between egocentric and exocentric worlds in video generation. Building upon advanced video diffusion transformers, WorldWander integrates (i) In-Context Perspective Alignment and (ii) Collaborative Position Encoding to efficiently model cross-view synchronization. To further support our task, we curate EgoExo-8K, a large-scale dataset containing synchronized egocentric-exocentric triplets from both synthetic and real-world scenarios. Experiments demonstrate that WorldWander achieves superior perspective synchronization, character consistency, and generalization, setting a new benchmark for egocentric-exocentric video translation.