Multi-task Learning for Camera Calibration

TL;DR

This paper introduces a novel multi-task learning approach using neural networks to jointly estimate intrinsic and extrinsic camera parameters from image pairs, outperforming traditional methods on several parameters.

Contribution

It presents the first multi-task learning framework incorporating mathematical camera models for joint intrinsic and extrinsic parameter estimation.

Findings

Outperforms conventional and deep learning methods on 6 out of 10 parameters

Introduces a new dataset from CARLA Simulator for camera calibration

Demonstrates effectiveness on both real and synthetic data

Abstract

For a number of tasks, such as 3D reconstruction, robotic interface, autonomous driving, etc., camera calibration is essential. In this study, we present a unique method for predicting intrinsic (principal point offset and focal length) and extrinsic (baseline, pitch, and translation) properties from a pair of images. We suggested a novel method where camera model equations are represented as a neural network in a multi-task learning framework, in contrast to existing methods, which build a comprehensive solution. By reconstructing the 3D points using a camera model neural network and then using the loss in reconstruction to obtain the camera specifications, this innovative camera projection loss (CPL) method allows us that the desired parameters should be estimated. As far as we are aware, our approach is the first one that uses an approach to multi-task learning that includes mathematical formulas in a framework for learning to estimate camera parameters to predict both the extrinsic and intrinsic parameters jointly. Additionally, we provided a new dataset named as CVGL Camera Calibration Dataset [1] which has been collected using the CARLA Simulator [2]. Actually, we show that our suggested strategy out performs both conventional methods and methods based on deep learning on 6 out of 10 parameters that were assessed using both real and synthetic data. Our code and generated dataset are available at https://github.com/thanif/Camera-Calibration-through-Camera-Projection-Loss.