EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

TL;DR

EgoMoD is a novel approach that predicts comprehensive future maps of environmental motion dynamics from short egocentric videos, enabling better long-term navigation planning in dynamic settings.

Contribution

It introduces a method to learn environment-wide motion tendencies from local observations using a video- and pose-conditioned architecture trained with privileged supervision.

Findings

Accurately predicts future motion maps in simulated environments.

Demonstrates zero-shot transferability to real-world systems.

Outperforms baseline methods in limited observability scenarios.

Abstract

Efficient navigation in dynamic environments requires anticipating how motion patterns evolve beyond the robot's immediate perceptual range, enabling preemptive rather than purely reactive planning in crowded scenes. Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning, but constructing them traditionally requires global environment observations over extended periods of time. We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation. Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision, allowing local observations to serve as predictive signals of global motion structure. Thanks to this, we offer the capacity to forecast future motion dynamics over the whole environment rather than merely extend past patterns in the robot's field of view. Experiments in large simulated environments show that EgoMoD accurately predicts future MoDs under limited observability, while evaluation with real images showcases its zero-shot transferability to real systems.