MamV2XCalib: V2X-based Target-less Infrastructure Camera Calibration with State Space Model

TL;DR

MamV2XCalib introduces a novel V2X-based infrastructure camera calibration method utilizing vehicle-side LiDAR, enabling automatic, targetless calibration in real-world scenarios without manual intervention or reference objects.

Contribution

This work presents the first V2X-based infrastructure camera calibration method that leverages vehicle LiDAR data, improving robustness and reducing manual effort compared to prior approaches.

Findings

Achieves more stable calibration performance in V2X scenarios.

Demonstrates effectiveness on real-world datasets.

Reduces calibration parameters needed for V2X environments.

Abstract

As cooperative systems that leverage roadside cameras to assist autonomous vehicle perception become increasingly widespread, large-scale precise calibration of infrastructure cameras has become a critical issue. Traditional manual calibration methods are often time-consuming, labor-intensive, and may require road closures. This paper proposes MamV2XCalib, the first V2X-based infrastructure camera calibration method with the assistance of vehicle-side LiDAR. MamV2XCalib only requires autonomous vehicles equipped with LiDAR to drive near the cameras to be calibrated in the infrastructure, without the need for specific reference objects or manual intervention. We also introduce a new targetless LiDAR-camera calibration method, which combines multi-scale features and a 4D correlation volume to estimate the correlation between vehicle-side point clouds and roadside images. We model the temporal information and estimate the rotation angles with Mamba, effectively addressing calibration failures in V2X scenarios caused by defects in the vehicle-side data (such as occlusions) and large differences in viewpoint. We evaluate MamV2XCalib on the V2X-Seq and TUMTraf-V2X real-world datasets, demonstrating the effectiveness and robustness of our V2X-based automatic calibration approach. Compared to previous LiDAR-camera methods designed for calibration on one car, our approach achieves better and more stable calibration performance in V2X scenarios with fewer parameters. The code is available at https://github.com/zhuyaoye/MamV2XCalib.