Learnable Differencing Center for Nighttime Depth Perception

TL;DR

This paper introduces LDCNet, a novel framework for nighttime depth perception that enhances image quality and reduces illumination effects using learnable differencing techniques, achieving state-of-the-art results.

Contribution

LDCNet employs RICD and IAICD modules to explicitly estimate illumination and adaptively enhance nighttime images for improved depth perception.

Findings

LDCNet outperforms existing methods on nighttime depth completion benchmarks.

The proposed differencing modules effectively handle varying illumination conditions.

LDCNet achieves state-of-the-art accuracy in nighttime depth estimation.

Abstract

Depth completion is the task of recovering dense depth maps from sparse ones, usually with the help of color images. Existing image-guided methods perform well on daytime depth perception self-driving benchmarks, but struggle in nighttime scenarios with poor visibility and complex illumination. To address these challenges, we propose a simple yet effective framework called LDCNet. Our key idea is to use Recurrent Inter-Convolution Differencing (RICD) and Illumination-Affinitive Intra-Convolution Differencing (IAICD) to enhance the nighttime color images and reduce the negative effects of the varying illumination, respectively. RICD explicitly estimates global illumination by differencing two convolutions with different kernels, treating the small-kernel-convolution feature as the center of the large-kernel-convolution feature in a new perspective. IAICD softly alleviates local relative light intensity by differencing a single convolution, where the center is dynamically aggregated based on neighboring pixels and the estimated illumination map in RICD. On both nighttime depth completion and depth estimation tasks, extensive experiments demonstrate the effectiveness of our LDCNet, reaching the state of the art.