UniCal: Unified Neural Sensor Calibration

TL;DR

UniCal introduces a scalable, cost-effective neural framework for calibrating autonomous vehicle sensors using differentiable scene representations, eliminating the need for fiducials and enabling large fleet deployment.

Contribution

It presents a unified, differentiable calibration method that jointly learns sensor calibration and scene representation without fiducials, improving scalability and efficiency.

Findings

Outperforms existing calibration methods in accuracy.

Reduces calibration costs and operational overhead.

Enables large-scale autonomous vehicle deployment.

Abstract

Self-driving vehicles (SDVs) require accurate calibration of LiDARs and cameras to fuse sensor data accurately for autonomy. Traditional calibration methods typically leverage fiducials captured in a controlled and structured scene and compute correspondences to optimize over. These approaches are costly and require substantial infrastructure and operations, making it challenging to scale for vehicle fleets. In this work, we propose UniCal, a unified framework for effortlessly calibrating SDVs equipped with multiple LiDARs and cameras. Our approach is built upon a differentiable scene representation capable of rendering multi-view geometrically and photometrically consistent sensor observations. We jointly learn the sensor calibration and the underlying scene representation through differentiable volume rendering, utilizing outdoor sensor data without the need for specific calibration fiducials. This "drive-and-calibrate" approach significantly reduces costs and operational overhead compared to existing calibration systems, enabling efficient calibration for large SDV fleets at scale. To ensure geometric consistency across observations from different sensors, we introduce a novel surface alignment loss that combines feature-based registration with neural rendering. Comprehensive evaluations on multiple datasets demonstrate that UniCal outperforms or matches the accuracy of existing calibration approaches while being more efficient, demonstrating the value of UniCal for scalable calibration.