Self-Supervised Camera Self-Calibration from Video

TL;DR

This paper presents a self-supervised learning method for calibrating various camera types directly from raw videos, improving depth estimation accuracy and efficiency without manual calibration.

Contribution

The authors introduce a novel learning algorithm that regresses per-sequence camera calibration parameters for diverse camera geometries from raw video data.

Findings

Achieves sub-pixel reprojection error in calibration

Outperforms existing learning-based calibration methods

Enhances depth estimation accuracy on EuRoC dataset

Abstract

Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams. In practice, calibration is a laborious procedure requiring specialized data collection and careful tuning. This process must be repeated whenever the parameters of the camera change, which can be a frequent occurrence for mobile robots and autonomous vehicles. In contrast, self-supervised depth and ego-motion estimation approaches can bypass explicit calibration by inferring per-frame projection models that optimize a view synthesis objective. In this paper, we extend this approach to explicitly calibrate a wide range of cameras from raw videos in the wild. We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models. Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods. We validate our approach on a wide variety of camera geometries, including perspective, fisheye, and catadioptric. Finally, we show that our approach leads to improvements in the downstream task of depth estimation, achieving state-of-the-art results on the EuRoC dataset with greater computational efficiency than contemporary methods.