EgoTraj: Real-World Egocentric Human Trajectory Dataset for Multimodal Prediction

TL;DR

EgoTraj is a new multimodal egocentric human trajectory dataset collected in real-world urban environments, enabling improved prediction models for applications like robotics and navigation.

Contribution

The paper introduces EgoTraj, a comprehensive dataset with synchronized multimodal data capturing long-horizon, self-directed navigation in diverse urban settings.

Findings

Benchmarking shows state-of-the-art methods benefit from the dataset.

Ablation studies reveal the importance of gaze, scene, and motion cues.

The dataset enhances AR perception, navigation, and assistive system development.

Abstract

Accurately forecasting human trajectories from an egocentric perspective plays a central role in applications such as humanoid robotics, wearable sensing systems, and assistive navigation. However, progress in this direction remains limited due to the scarcity of egocentric trajectory datasets collected in real-world environments. Addressing this need, we introduce EgoTraj, an egocentric multimodal open dataset recorded using Meta Quest Pro (MQPro). EgoTraj contains 75 sequences of human navigation collected from multiple MQPro wearers in real-world urban environments. Each recording provides synchronized RGB video along with ground-truth data, including continuous time-synchronized 6-degree-of-freedom head poses, per-frame 3D eye gaze vectors, scene annotations. To the best of our knowledge, EgoTraj differs from typical egocentric trajectory datasets by capturing long-horizon, self-directed navigation across diverse urban routes with broad participant diversity. To demonstrate the potential of the dataset, we benchmark several state-of-the-art methods for egocentric trajectory prediction and conduct ablation studies to analyze the contributions of gaze, scene, and motion cues. The results highlight the utility of EgoTraj for AR-based perception, navigation, and assistive systems. The EgoTraj dataset, code, and EgoViz Dashboard are publicly available at https://github.com/yehiahmad/EgoTraj.