Spatiotemporal Calibration of 3D Millimetre-Wavelength Radar-Camera Pairs

TL;DR

This paper presents a targetless, in-situ spatiotemporal calibration algorithm for 3D radar-camera pairs in autonomous vehicles, enabling reliable long-term sensor fusion without specialized targets.

Contribution

The proposed method achieves accurate, targetless calibration of radar and camera sensors in arbitrary environments, matching target-based methods' performance.

Findings

Accurate calibration achieved in real-world scenarios.

Method matches performance of target-based calibration.

Operates without specialized infrastructure.

Abstract

Autonomous vehicles (AVs) fuse data from multiple sensors and sensing modalities to impart a measure of robustness when operating in adverse conditions. Radars and cameras are popular choices for use in sensor fusion; although radar measurements are sparse in comparison to camera images, radar scans penetrate fog, rain, and snow. However, accurate sensor fusion depends upon knowledge of the spatial transform between the sensors and any temporal misalignment that exists in their measurement times. During the life cycle of an AV, these calibration parameters may change, so the ability to perform in-situ spatiotemporal calibration is essential to ensure reliable long-term operation. State-of-the-art 3D radar-camera spatiotemporal calibration algorithms require bespoke calibration targets that are not readily available in the field. In this paper, we describe an algorithm for targetless spatiotemporal calibration that does not require specialized infrastructure. Our approach leverages the ability of the radar unit to measure its own ego-velocity relative to a fixed, external reference frame. We analyze the identifiability of the spatiotemporal calibration problem and determine the motions necessary for calibration. Through a series of simulation studies, we characterize the sensitivity of our algorithm to measurement noise. Finally, we demonstrate accurate calibration for three real-world systems, including a handheld sensor rig and a vehicle-mounted sensor array. Our results show that we are able to match the performance of an existing, target-based method, while calibrating in arbitrary, infrastructure-free environments.