Depth Estimation from Monocular Images and Sparse Radar Data

TL;DR

This paper presents a deep learning approach to improve depth estimation by fusing monocular images with sparse Radar data, addressing noise issues and outperforming existing methods on the nuScenes dataset.

Contribution

The paper introduces a novel fusion method for monocular images and Radar data, specifically tackling Radar noise challenges and demonstrating superior performance.

Findings

Our method outperforms existing fusion techniques.

Radar noise significantly impacts fusion accuracy.

Ablation studies confirm the effectiveness of each component.

Abstract

In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We give a comprehensive study of the fusion between RGB images and Radar measurements from different aspects and proposed a working solution based on the observations. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods developed for LiDAR data and images to the new fusion problem between Radar data and images. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions. Extensive experiments demonstrate that our method outperforms existing fusion methods. We also provide detailed ablation studies to show the effectiveness of each component in our method.