A Synthetic Benchmarking Pipeline to Compare Camera Calibration Algorithms

TL;DR

SynthCal is a synthetic benchmarking pipeline that generates diverse calibration datasets to accurately evaluate camera calibration algorithms' performance across different patterns, camera types, and environmental conditions.

Contribution

The paper introduces SynthCal, a novel synthetic dataset and benchmarking pipeline for comprehensive evaluation of camera calibration algorithms in various settings.

Findings

SynthCal effectively evaluates calibration algorithms with diverse patterns and conditions.

Calibration accuracy varies significantly with pattern choice and environmental factors.

SynthCal provides a reliable ground truth for calibration performance assessment.

Abstract

Accurate camera calibration is crucial for various computer vision applications. However, measuring calibration accuracy in the real world is challenging due to the lack of datasets with ground truth to evaluate them. In this paper, we present SynthCal, a synthetic camera calibration benchmarking pipeline that generates images of calibration patterns to measure and enable accurate quantification of calibration algorithm performance in camera parameter estimation. We present a SynthCal generated calibration dataset with four common patterns, two camera types, and two environments with varying view, distortion, lighting, and noise levels for both monocular and multi-camera systems. The dataset evaluates both single and multi-view calibration algorithms by measuring re-projection and root-mean-square errors for identical patterns and camera settings. Additionally, we analyze the significance of different patterns using different calibration configurations. The experimental results demonstrate the effectiveness of SynthCal in evaluating various calibration algorithms and patterns.