Robust Image Stitching with Optimal Plane

TL;DR

RopStitch is an unsupervised deep image stitching framework that enhances robustness and naturalness by combining dual-branch feature extraction with optimal plane estimation to handle diverse real-world scenes effectively.

Contribution

The paper introduces a novel dual-branch architecture and a virtual optimal plane concept to improve robustness and content preservation in image stitching.

Findings

Outperforms existing methods in robustness across various datasets

Effectively balances content alignment and structural preservation

Achieves high naturalness in stitched images

Abstract

We present \textit{RopStitch}, an unsupervised deep image stitching framework with both robustness and naturalness. To ensure the robustness of \textit{RopStitch}, we propose to incorporate the universal prior of content perception into the image stitching model by a dual-branch architecture. It separately captures coarse and fine features and integrates them to achieve highly generalizable performance across diverse unseen real-world scenes. Concretely, the dual-branch model consists of a pretrained branch to capture semantically invariant representations and a learnable branch to extract fine-grained discriminative features, which are then merged into a whole by a controllable factor at the correlation level. Besides, considering that content alignment and structural preservation are often contradictory to each other, we propose a concept of virtual optimal planes to relieve this conflict. To this end, we model this problem as a process of estimating homography decomposition coefficients, and design an iterative coefficient predictor and minimal semantic distortion constraint to identify the optimal plane. This scheme is finally incorporated into \textit{RopStitch} by warping both views onto the optimal plane bidirectionally. Extensive experiments across various datasets demonstrate that \textit{RopStitch} significantly outperforms existing methods, particularly in scene robustness and content naturalness. The code is available at {\color{red}https://github.com/MmelodYy/RopStitch}.