MultiEgo: A Multi-View Egocentric Video Dataset for 4D Scene Reconstruction

TL;DR

MultiEgo introduces the first multi-view egocentric dataset for 4D dynamic scene reconstruction, enabling research in holographic social interaction documentation with synchronized multi-view videos and accurate pose data.

Contribution

This paper presents a novel multi-view egocentric dataset with synchronized videos and pose annotations, filling a gap in dynamic scene reconstruction research.

Findings

Dataset enables advanced 4D scene reconstruction research.

High temporal synchronization accuracy achieved.

Validated for free-viewpoint video applications.

Abstract

Multi-view egocentric dynamic scene reconstruction holds significant research value for applications in holographic documentation of social interactions. However, existing reconstruction datasets focus on static multi-view or single-egocentric view setups, lacking multi-view egocentric datasets for dynamic scene reconstruction. Therefore, we present MultiEgo, the first multi-view egocentric dataset for 4D dynamic scene reconstruction. The dataset comprises five canonical social interaction scenes: meetings, performances, and a presentation. Each scene provides five authentic egocentric videos captured by participants wearing AR glasses. We design a hardware-based data acquisition system and processing pipeline, achieving sub-millisecond temporal synchronization across views, coupled with accurate pose annotations. Experiment validation demonstrates the practical utility and effectiveness of our dataset for free-viewpoint video (FVV) applications, establishing MultiEgo as a foundational resource for advancing multi-view egocentric dynamic scene reconstruction research.