Extrinsic Camera Calibration with Semantic Segmentation

TL;DR

This paper introduces an automated extrinsic camera calibration method that uses semantic segmentation and lidar data to accurately determine camera parameters without manual intervention, suitable for vehicle and infrastructure sensors.

Contribution

The novel approach combines semantic segmentation with lidar-to-camera registration to automate calibration, eliminating the need for camera motion or manual measurements.

Findings

Achieves low calibration error in simulated and real-world tests.

Works with both infrastructure and vehicle-mounted sensors.

Does not require camera movement during calibration.

Abstract

Monocular camera sensors are vital to intelligent vehicle operation and automated driving assistance and are also heavily employed in traffic control infrastructure. Calibrating the monocular camera, though, is time-consuming and often requires significant manual intervention. In this work, we present an extrinsic camera calibration approach that automatizes the parameter estimation by utilizing semantic segmentation information from images and point clouds. Our approach relies on a coarse initial measurement of the camera pose and builds on lidar sensors mounted on a vehicle with high-precision localization to capture a point cloud of the camera environment. Afterward, a mapping between the camera and world coordinate spaces is obtained by performing a lidar-to-camera registration of the semantically segmented sensor data. We evaluate our method on simulated and real-world data to demonstrate low error measurements in the calibration results. Our approach is suitable for infrastructure sensors as well as vehicle sensors, while it does not require motion of the camera platform.