Dark3R: Learning Structure from Motion in the Dark

TL;DR

Dark3R is a novel framework that enables structure from motion in extremely low-light conditions by adapting large-scale 3D models through a teacher-student approach, trained on noisy and clean raw image pairs without 3D supervision.

Contribution

It introduces a low-light SfM method using a teacher-student distillation on raw images, along with a new dataset for training and evaluation in dark environments.

Findings

Achieves state-of-the-art SfM performance in low-SNR regimes.

Demonstrates effective novel view synthesis in dark conditions.

Introduces a new dataset with 42,000 multi-view raw images.

Abstract

We introduce Dark3R, a framework for structure from motion in the dark that operates directly on raw images with signal-to-noise ratios (SNRs) below $-4$ dB -- a regime where conventional feature- and learning-based methods break down. Our key insight is to adapt large-scale 3D foundation models to extreme low-light conditions through a teacher--student distillation process, enabling robust feature matching and camera pose estimation in low light. Dark3R requires no 3D supervision; it is trained solely on noisy--clean raw image pairs, which can be either captured directly or synthesized using a simple Poisson--Gaussian noise model applied to well-exposed raw images. To train and evaluate our approach, we introduce a new, exposure-bracketed dataset that includes $\sim$42,000 multi-view raw images with ground-truth 3D annotations, and we demonstrate that Dark3R achieves state-of-the-art structure from motion in the low-SNR regime. Further, we demonstrate state-of-the-art novel view synthesis in the dark using Dark3R's predicted poses and a coarse-to-fine radiance field optimization procedure.