A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation

TL;DR

This paper introduces a new online camera calibration method called fR that leverages a novel dataset and evaluation of perspective deviation measures to improve the reliability and accuracy of estimating focal length and rotation in real-time.

Contribution

It presents a novel dataset, evaluates deviation measures, and develops the fR system that outperforms previous methods in online camera calibration accuracy and reliability.

Findings

fR reduces calibration error significantly

Choice of deviation measure impacts reliability

Uncertainty estimation improves recalibration decisions

Abstract

Linear perspectivecues deriving from regularities of the built environment can be used to recalibrate both intrinsic and extrinsic camera parameters online, but these estimates can be unreliable due to irregularities in the scene, uncertainties in line segment estimation and background clutter. Here we address this challenge through four initiatives. First, we use the PanoContext panoramic image dataset [27] to curate a novel and realistic dataset of planar projections over a broad range of scenes, focal lengths and camera poses. Second, we use this novel dataset and the YorkUrbanDB [4] to systematically evaluate the linear perspective deviation measures frequently found in the literature and show that the choice of deviation measure and likelihood model has a huge impact on reliability. Third, we use these findings to create a novel system for online camera calibration we call fR, and show that it outperforms the prior state of the art, substantially reducing error in estimated camera rotation and focal length. Our fourth contribution is a novel and efficient approach to estimating uncertainty that can dramatically improve online reliability for performance-critical applications by strategically selecting which frames to use for recalibration.