# Targetless LiDAR–camera extrinsic calibration via semantic distribution alignment

## TL;DR

This paper introduces a new method for calibrating LiDAR and camera sensors without targets, using semantic alignment to improve stability and accuracy in robotic systems.

## Contribution

The novel approach uses semantic distribution alignment with an anchor-fixed pixel measure and direction-aware weighting for robust targetless LiDAR-camera calibration.

## Key findings

- The method reliably converges even with large initial perturbations.
- It provides stable extrinsic estimates on the KITTI Odometry dataset.
- The approach improves robustness in challenging outdoor scenes.

## Abstract

LiDAR–camera fusion systems are widely used in robotic localization and perception, where accurate extrinsic calibration is crucial for multi-sensor fusion. During long-term operation, extrinsic parameters can drift due to vibration and other disturbances, while target-based recalibration is inconvenient in the field and targetless approaches often suffer from highly non-convex objectives and limited robustness in challenging outdoor scenes.

We propose a targetless LiDAR–camera extrinsic calibration method by minimizing a semantic distribution consistency risk on SE(3). We align semantic probability distributions from the two sensing modalities in the image domain and freeze the pixel sampling measure at an anchor pose, so that pixel weighting no longer depends on the current extrinsic estimate and the objective landscape remains stable during optimization. On top of this anchor-fixed measure, we introduce a direction-aware weighting strategy that emphasizes pixels sensitive to yaw perturbations, improving the conditioning of rotation estimation. We further use a globally balanced Jensen–Shannon divergence to mitigate semantic class imbalance and enhance robustness.

Experiments on the KITTI Odometry dataset show that the proposed method reliably converges from substantial initial perturbations and yields stable extrinsic estimates.

The results indicate that the method is promising for maintaining long-term LiDAR–camera calibration in real-world robotic systems.