Image Stitching and Rectification for Hand-Held Cameras

TL;DR

This paper introduces a novel differential homography model for rolling shutter cameras, enabling improved image stitching and rectification for hand-held videos with camera motion constraints.

Contribution

It presents a new differential homography for RS cameras and an RS-aware spatially-varying homography field, enhancing stitching and rectification accuracy.

Findings

Outperforms state-of-the-art methods in RS image stitching

Provides superior rectification for hand-held camera images

Effective for videos with limited inter-frame motion

Abstract

In this paper, we derive a new differential homography that can account for the scanline-varying camera poses in Rolling Shutter (RS) cameras, and demonstrate its application to carry out RS-aware image stitching and rectification at one stroke. Despite the high complexity of RS geometry, we focus in this paper on a special yet common input -- two consecutive frames from a video stream, wherein the inter-frame motion is restricted from being arbitrarily large. This allows us to adopt simpler differential motion model, leading to a straightforward and practical minimal solver. To deal with non-planar scene and camera parallax in stitching, we further propose an RS-aware spatially-varying homography field in the principle of As-Projective-As-Possible (APAP). We show superior performance over state-of-the-art methods both in RS image stitching and rectification, especially for images captured by hand-held shaking cameras.