Toward a Low-Cost Perception System in Autonomous Vehicles: A Spectrum Learning Approach

TL;DR

This paper introduces a cost-effective perception system for autonomous vehicles that combines radar and camera data using a novel spectrum learning approach, significantly improving depth map quality and outperforming current methods.

Contribution

The paper proposes a new spectrum learning algorithm that unifies radar and camera data into a common subspace, enhancing depth estimation in autonomous driving.

Findings

Outperforms SOTA by 27.95% in Unidirectional Chamfer Distance

Effectively sharpens radar output in complex environments

Leverages high-resolution camera images to improve radar depth maps

Abstract

We present a cost-effective new approach for generating denser depth maps for Autonomous Driving (AD) and Autonomous Vehicles (AVs) by integrating the images obtained from deep neural network (DNN) 4D radar detectors with conventional camera RGB images. Our approach introduces a novel pixel positional encoding algorithm inspired by Bartlett's spatial spectrum estimation technique. This algorithm transforms both radar depth maps and RGB images into a unified pixel image subspace called the Spatial Spectrum, facilitating effective learning based on their similarities and differences. Our method effectively leverages high-resolution camera images to train radar depth map generative models, addressing the limitations of conventional radar detectors in complex vehicular environments, thus sharpening the radar output. We develop spectrum estimation algorithms tailored for radar depth maps and RGB images, a comprehensive training framework for data-driven generative models, and a camera-radar deployment scheme for AV operation. Our results demonstrate that our approach also outperforms the state-of-the-art (SOTA) by 27.95% in terms of Unidirectional Chamfer Distance (UCD).