Minimal Solutions for Panoramic Stitching Given Gravity Prior

TL;DR

This paper introduces minimal solutions for panoramic stitching that leverage gravity priors from IMU sensors, reducing the problem to 1-DOF rotation, and demonstrates improved accuracy and efficiency over existing methods.

Contribution

It presents new minimal solvers for panoramic stitching using gravity priors, applicable to various camera configurations with unknown focal length and distortion.

Findings

Outperforms state-of-the-art in accuracy and speed

Validated on synthetic and real datasets with over 500k image pairs

Effective for cameras with pure rotation and known gravity direction

Abstract

When capturing panoramas, people tend to align their cameras with the vertical axis, i.e., the direction of gravity. Moreover, modern devices, such as smartphones and tablets, are equipped with an IMU (Inertial Measurement Unit) that can measure the gravity vector accurately. Using this prior, the y-axes of the cameras can be aligned or assumed to be already aligned, reducing their relative orientation to 1-DOF (degree of freedom). Exploiting this assumption, we propose new minimal solutions to panoramic image stitching of images taken by cameras with coinciding optical centers, i.e., undergoing pure rotation. We consider four practical camera configurations, assuming unknown fixed or varying focal length with or without radial distortion. The solvers are tested both on synthetic scenes and on more than 500k real image pairs from the Sun360 dataset and from scenes captured by us using two smartphones equipped with IMUs. It is shown, that they outperform the state-of-the-art both in terms of accuracy and processing time.