Self-calibration-based Approach to Critical Motion Sequences of Rolling-shutter Structure from Motion

TL;DR

This paper introduces a self-calibration approach to analyze and resolve critical motion sequences in rolling-shutter structure from motion, modeling distortion with internal parameters and reformulating as a self-calibration problem.

Contribution

It presents a novel self-calibration framework for rolling-shutter SfM, deriving a general representation of critical motion sequences and addressing degeneracies.

Findings

The method explains previously observed degeneracies.

The approach effectively resolves degeneracies in experiments.

Theoretical results align with experimental observations.

Abstract

In this paper we consider critical motion sequences (CMSs) of rolling-shutter (RS) SfM. Employing an RS camera model with linearized pure rotation, we show that the RS distortion can be approximately expressed by two internal parameters of an "imaginary" camera plus one-parameter nonlinear transformation similar to lens distortion. We then reformulate the problem as self-calibration of the imaginary camera, in which its skew and aspect ratio are unknown and varying in the image sequence. In the formulation, we derive a general representation of CMSs. We also show that our method can explain the CMS that was recently reported in the literature, and then present a new remedy to deal with the degeneracy. Our theoretical results agree well with experimental results; it explains degeneracies observed when we employ naive bundle adjustment, and how they are resolved by our method.