Object-centered image stitching

TL;DR

This paper introduces an object-centered approach to image stitching that improves seam placement by considering object detection, leading to more realistic results and better handling of occlusions.

Contribution

It proposes a novel method that incorporates object detection into seam finding, enhancing stitching quality and occlusion detection.

Findings

More realistic stitching results on challenging images

Ability to detect non-recoverable occlusions

A new evaluation metric for stitching quality

Abstract

Image stitching is typically decomposed into three phases: registration, which aligns the source images with a common target image; seam finding, which determines for each target pixel the source image it should come from; and blending, which smooths transitions over the seams. As described in [1], the seam finding phase attempts to place seams between pixels where the transition between source images is not noticeable. Here, we observe that the most problematic failures of this approach occur when objects are cropped, omitted, or duplicated. We therefore take an object-centered approach to the problem, leveraging recent advances in object detection [2,3,4]. We penalize candidate solutions with this class of error by modifying the energy function used in the seam finding stage. This produces substantially more realistic stitching results on challenging imagery. In addition, these methods can be used to determine when there is non-recoverable occlusion in the input data, and also suggest a simple evaluation metric that can be used to evaluate the output of stitching algorithms.