CTRL-C: Camera calibration TRansformer with Line-Classification

TL;DR

CTRL-C introduces a transformer-based neural network that accurately estimates camera parameters from a single image by leveraging line classification and global image structure, outperforming previous methods on standard benchmarks.

Contribution

The paper presents a novel end-to-end transformer architecture with line classification for improved single image camera calibration.

Findings

CTRL-C outperforms state-of-the-art methods on Google Street View dataset.

The line classification auxiliary task enhances geometric feature extraction.

Transformer architecture effectively captures global image structure for calibration.

Abstract

Single image camera calibration is the task of estimating the camera parameters from a single input image, such as the vanishing points, focal length, and horizon line. In this work, we propose Camera calibration TRansformer with Line-Classification (CTRL-C), an end-to-end neural network-based approach to single image camera calibration, which directly estimates the camera parameters from an image and a set of line segments. Our network adopts the transformer architecture to capture the global structure of an image with multi-modal inputs in an end-to-end manner. We also propose an auxiliary task of line classification to train the network to extract the global geometric information from lines effectively. Our experiments demonstrate that CTRL-C outperforms the previous state-of-the-art methods on the Google Street View and SUN360 benchmark datasets.