EgoForge: Goal-Directed Egocentric World Simulator

TL;DR

EgoForge is a novel egocentric world simulation framework that generates coherent first-person videos from minimal inputs, improving goal alignment and scene consistency through trajectory-guided diffusion refinement.

Contribution

The paper introduces EgoForge, a goal-directed egocentric world simulator that produces realistic video rollouts from limited inputs, with a new diffusion-based refinement method for enhanced temporal and semantic coherence.

Findings

EgoForge outperforms baseline models in semantic alignment and scene stability.

The approach demonstrates robustness in real-world smart-glasses experiments.

Trajectory-guided diffusion improves goal completion and perceptual fidelity.

Abstract

Generative world models have shown promise for simulating dynamic environments, yet egocentric video remains challenging due to rapid viewpoint changes, frequent hand-object interactions, and goal-directed procedures whose evolution depends on latent human intent. Existing approaches either focus on hand-centric instructional synthesis with limited scene evolution, perform static view translation without modeling action dynamics, or rely on dense supervision, such as camera trajectories, long video prefixes, synchronized multicamera capture, etc. In this work, we introduce EgoForge, an egocentric goal-directed world simulator that generates coherent, first-person video rollouts from minimal static inputs: a single egocentric image, a high-level instruction, and an optional auxiliary exocentric view. To improve intent alignment and temporal consistency, we propose VideoDiffusionNFT, a trajectory-level reward-guided refinement that optimizes goal completion, temporal causality, scene consistency, and perceptual fidelity during diffusion sampling. Extensive experiments show EgoForge achieves consistent gains in semantic alignment, geometric stability, and motion fidelity over strong baselines, and robust performance in real-world smart-glasses experiments.