LED: Light Enhanced Depth Estimation at Night

TL;DR

This paper introduces Light Enhanced Depth (LED), a cost-effective method that uses vehicle headlights to significantly improve nighttime depth estimation across various models and datasets, enhancing scene understanding.

Contribution

The paper presents LED, a novel approach leveraging vehicle headlights to boost low-light depth estimation, and releases a new synthetic nighttime dataset for training and evaluation.

Findings

LED improves depth estimation accuracy in low-light conditions.

Performance gains observed across multiple architectures.

Enhanced scene understanding beyond illuminated areas.

Abstract

Nighttime camera-based depth estimation is a highly challenging task, especially for autonomous driving applications, where accurate depth perception is essential for ensuring safe navigation. Models trained on daytime data often fail in the absence of precise but costly LiDAR. Even vision foundation models trained on large amounts of data are unreliable in low-light conditions. In this work, we aim to improve the reliability of perception systems at night time. To this end, we introduce Light Enhanced Depth (LED), a novel, cost-effective approach that significantly improves depth estimation in low-light environments by harnessing a pattern projected by high definition headlights available in modern vehicles. LED leads to significant performance boosts across multiple depth-estimation architectures (encoder-decoder, Adabins, DepthFormer, Depth Anything V2) both on synthetic and real datasets. Furthermore, increased performances beyond illuminated areas reveal a holistic enhancement in scene understanding. Finally, we release the Nighttime Synthetic Drive Dataset, a synthetic and photo-realistic nighttime dataset, which comprises 49,990 comprehensively annotated images.