Parallax-tolerant Image Stitching via Segmentation-guided Multi-homography Warping

TL;DR

This paper introduces a segmentation-guided multi-homography warping method for image stitching that effectively handles large parallax, achieving superior alignment accuracy over existing techniques.

Contribution

It proposes a novel approach combining image segmentation with multi-homography fitting to improve stitching quality in parallax-rich scenarios.

Findings

Achieves the best alignment accuracy on public datasets.

Outperforms state-of-the-art image stitching methods.

Demonstrates robustness to large parallax in image pairs.

Abstract

Large parallax between images is an intractable issue in image stitching. Various warping-based methods are proposed to address it, yet the results are unsatisfactory. In this paper, we propose a novel image stitching method using multi-homography warping guided by image segmentation. Specifically, we leverage the Segment Anything Model to segment the target image into numerous contents and partition the feature points into multiple subsets via the energy-based multi-homography fitting algorithm. The multiple subsets of feature points are used to calculate the corresponding multiple homographies. For each segmented content in the overlapping region, we select its best-fitting homography with the lowest photometric error. For each segmented content in the non-overlapping region, we calculate a weighted combination of the linearized homographies. Finally, the target image is warped via the best-fitting homographies to align with the reference image, and the final panorama is generated via linear blending. Comprehensive experimental results on the public datasets demonstrate that our method provides the best alignment accuracy by a large margin, compared with the state-of-the-art methods. The source code is available at https://github.com/tlliao/multi-homo-warp.