Deep Learning for Camera Calibration and Beyond: A Survey

TL;DR

This survey reviews a decade of learning-based camera calibration methods, analyzing their strengths, limitations, and datasets, and introduces a new comprehensive benchmark dataset for evaluating these techniques.

Contribution

It provides the first extensive survey of learning-based camera calibration over 10 years, and introduces a unified benchmark dataset for method evaluation.

Findings

Various learning strategies and models have been explored for camera calibration.

A new holistic dataset with synthetic and real data has been collected for benchmarking.

The survey highlights challenges and future research directions in this field.

Abstract

Camera calibration involves estimating camera parameters to infer geometric features from captured sequences, which is crucial for computer vision and robotics. However, conventional calibration is laborious and requires dedicated collection. Recent efforts show that learning-based solutions have the potential to be used in place of the repeatability works of manual calibrations. Among these solutions, various learning strategies, networks, geometric priors, and datasets have been investigated. In this paper, we provide a comprehensive survey of learning-based camera calibration techniques, by analyzing their strengths and limitations. Our main calibration categories include the standard pinhole camera model, distortion camera model, cross-view model, and cross-sensor model, following the research trend and extended applications. As there is no unified benchmark in this community, we collect a holistic calibration dataset that can serve as a public platform to evaluate the generalization of existing methods. It comprises both synthetic and real-world data, with images and videos captured by different cameras in diverse scenes. Toward the end of this paper, we discuss the challenges and provide further research directions. To our knowledge, this is the first survey for the learning-based camera calibration (spanned 10 years). The summarized methods, datasets, and benchmarks are available and will be regularly updated at https://github.com/KangLiao929/Awesome-Deep-Camera-Calibration.