EVEN: An Event-Based Framework for Monocular Depth Estimation at Adverse Night Conditions

TL;DR

This paper introduces an event-based framework combining RGB and event data, along with low-light enhancement, to improve monocular depth estimation in adverse night conditions, outperforming existing methods.

Contribution

It presents a novel integrated framework leveraging RGB and event modalities with a new dataset for accurate depth estimation at night.

Findings

Framework outperforms six baseline methods in adverse night conditions.

Event data's high dynamic range enhances depth estimation in low-light scenarios.

Combining RGB and event data improves robustness across various adverse weather conditions.

Abstract

Accurate depth estimation under adverse night conditions has practical impact and applications, such as on autonomous driving and rescue robots. In this work, we studied monocular depth estimation at night time in which various adverse weather, light, and different road conditions exist, with data captured in both RGB and event modalities. Event camera can better capture intensity changes by virtue of its high dynamic range (HDR), which is particularly suitable to be applied at adverse night conditions in which the amount of light is limited in the scene. Although event data can retain visual perception that conventional RGB camera may fail to capture, the lack of texture and color information of event data hinders its applicability to accurately estimate depth alone. To tackle this problem, we propose an event-vision based framework that integrates low-light enhancement for the RGB source, and exploits the complementary merits of RGB and event data. A dataset that includes paired RGB and event streams, and ground truth depth maps has been constructed. Comprehensive experiments have been conducted, and the impact of different adverse weather combinations on the performance of framework has also been investigated. The results have shown that our proposed framework can better estimate monocular depth at adverse nights than six baselines.