A Continuous-Time Approach for 3D Radar-to-Camera Extrinsic Calibration

TL;DR

This paper introduces a continuous-time algorithm for calibrating 3D radar and camera sensors in autonomous vehicles, leveraging radar velocity data without needing environmental retroreflectors, enhancing robustness in adverse weather.

Contribution

It presents a novel continuous-time calibration method for 3D radar-to-camera systems that does not require retroreflectors, unlike previous 2D radar calibration techniques.

Findings

Effective in synthetic and real-world tests

Does not require environmental retroreflectors

Improves robustness of sensor fusion in adverse weather

Abstract

Reliable operation in inclement weather is essential to the deployment of safe autonomous vehicles (AVs). Robustness and reliability can be achieved by fusing data from the standard AV sensor suite (i.e., lidars, cameras) with weather robust sensors, such as millimetre-wavelength radar. Critically, accurate sensor data fusion requires knowledge of the rigid-body transform between sensor pairs, which can be determined through the process of extrinsic calibration. A number of extrinsic calibration algorithms have been designed for 2D (planar) radar sensors - however, recently-developed, low-cost 3D millimetre-wavelength radars are set to displace their 2D counterparts in many applications. In this paper, we present a continuous-time 3D radar-to-camera extrinsic calibration algorithm that utilizes radar velocity measurements and, unlike the majority of existing techniques, does not require specialized radar retroreflectors to be present in the environment. We derive the observability properties of our formulation and demonstrate the efficacy of our algorithm through synthetic and real-world experiments.