Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network

TL;DR

This paper presents a deep learning approach that combines monocular images and sparse radar data to produce accurate dense depth maps, achieving state-of-the-art results in diverse lighting conditions.

Contribution

It introduces a novel preprocessing method for radar data, a late fusion approach for integrating radar with images, and demonstrates improved depth estimation performance.

Findings

Enhanced depth maps with reduced radar error

State-of-the-art accuracy on nuScenes dataset

Effective in both day and night scenes

Abstract

We integrate sparse radar data into a monocular depth estimation model and introduce a novel preprocessing method for reducing the sparseness and limited field of view provided by radar. We explore the intrinsic error of different radar modalities and show our proposed method results in more data points with reduced error. We further propose a novel method for estimating dense depth maps from monocular 2D images and sparse radar measurements using deep learning based on the deep ordinal regression network by Fu et al. Radar data are integrated by first converting the sparse 2D points to a height-extended 3D measurement and then including it into the network using a late fusion approach. Experiments are conducted on the nuScenes dataset. Our experiments demonstrate state-of-the-art performance in both day and night scenes.