Seeing in the Dark: Benchmarking Egocentric 3D Vision with the Oxford Day-and-Night Dataset

TL;DR

The Oxford Day-and-Night dataset provides a comprehensive egocentric benchmark for 3D vision tasks under diverse lighting conditions, enabling evaluation of novel view synthesis and relocalisation methods in realistic scenarios.

Contribution

It introduces a large-scale egocentric dataset with ground-truth 3D geometry and lighting variation, filling gaps in existing datasets for 3D vision research.

Findings

Dataset covers over 30 km of trajectories and 40,000 m² area.

Supports benchmarks for novel view synthesis and relocalisation.

Facilitates research in egocentric 3D vision under challenging lighting.

Abstract

We introduce Oxford Day-and-Night, a large-scale, egocentric dataset for novel view synthesis (NVS) and visual relocalisation under challenging lighting conditions. Existing datasets often lack crucial combinations of features such as ground-truth 3D geometry, wide-ranging lighting variation, and full 6DoF motion. Oxford Day-and-Night addresses these gaps by leveraging Meta ARIA glasses to capture egocentric video and applying multi-session SLAM to estimate camera poses, reconstruct 3D point clouds, and align sequences captured under varying lighting conditions, including both day and night. The dataset spans over 30 $\mathrm{km}$ of recorded trajectories and covers an area of 40,000 $\mathrm{m}^2$, offering a rich foundation for egocentric 3D vision research. It supports two core benchmarks, NVS and relocalisation, providing a unique platform for evaluating models in realistic and diverse environments.