Full-Velocity Radar Returns by Radar-Camera Fusion

TL;DR

This paper introduces a method to estimate full object velocities by fusing radar Doppler data with camera optical flow, improving velocity accuracy and radar data integration in dynamic scenes.

Contribution

It presents a closed-form solution for full velocity estimation and a neural network for radar-camera association, advancing radar data utilization in autonomous systems.

Findings

Significant improvement in velocity estimation accuracy.

Enhanced radar point accumulation in dynamic scenes.

Validated on nuScenes dataset with state-of-the-art results.

Abstract

A distinctive feature of Doppler radar is the measurement of velocity in the radial direction for radar points. However, the missing tangential velocity component hampers object velocity estimation as well as temporal integration of radar sweeps in dynamic scenes. Recognizing that fusing camera with radar provides complementary information to radar, in this paper we present a closed-form solution for the point-wise, full-velocity estimate of Doppler returns using the corresponding optical flow from camera images. Additionally, we address the association problem between radar returns and camera images with a neural network that is trained to estimate radar-camera correspondences. Experimental results on the nuScenes dataset verify the validity of the method and show significant improvements over the state-of-the-art in velocity estimation and accumulation of radar points.