Ego-1K -- A Large-Scale Multiview Video Dataset for Egocentric Vision

TL;DR

Ego-1K is a large-scale multiview egocentric video dataset designed to facilitate research in neural 3D video synthesis and dynamic scene understanding, especially for smart glasses with multiple cameras.

Contribution

The paper introduces Ego-1K, a novel dataset with synchronized multiview egocentric videos capturing hand interactions, enabling benchmarking and advancing egocentric scene reconstruction methods.

Findings

Existing 3D and 4D view synthesis methods face challenges with Ego-1K due to large disparities and motion.

The dataset highlights the need for improved algorithms to handle egocentric view complexities.

Ego-1K supports future research in egocentric vision and scene reconstruction.

Abstract

We present Ego-1K, a large-scale collection of time-synchronized egocentric multiview videos designed to advance neural 3D video synthesis and dynamic scene understanding. The dataset contains nearly 1,000 short egocentric videos captured with a custom rig with 12 synchronized cameras surrounding a 4-camera VR headset worn by the user. Scene content focuses on hand motions and hand-object interactions in different settings. We describe rig design, data processing, and calibration. Our dataset enables new ways to benchmark egocentric scene reconstruction methods, an important research area as smart glasses with multiple cameras become omnipresent. Our experiments demonstrate that our dataset presents unique challenges for existing 3D and 4D novel view synthesis methods due to large disparities and image motion caused by close dynamic objects and rig egomotion. Our dataset supports future research in this challenging domain. It is available at https://huggingface.co/datasets/facebook/ego-1k.