Natural Image Stitching Using Depth Maps

TL;DR

This paper introduces a novel image stitching method that leverages depth maps to improve alignment accuracy and view consistency in challenging non-planar scenes with parallax.

Contribution

It proposes a new approach combining robust feature matching, epipolar geometry, and optimal warping to enhance natural image stitching with depth information.

Findings

More accurate alignment in overlapping regions

View-consistent results in non-overlapping regions

Effective on challenging datasets

Abstract

Natural image stitching aims to create a single, natural-looking mosaic from overlapped images that capture the same 3D scene from different viewing positions. Challenges inevitably arise when the scene is non-planar and captured by handheld cameras since parallax is non-negligible in such cases. In this paper, we propose a novel image stitching method using depth maps, which generates accurate alignment mosaics against parallax. Firstly, we construct a robust fitting method to filter out the outliers in feature matches and estimate the epipolar geometry between input images. Then, we utilize epipolar geometry to establish pixel-to-pixel correspondences between the input images and render the warped images using the proposed optimal warping. In the rendering stage, we introduce several modules to solve the mapping artifacts in the warping results and generate the final mosaic. Experimental results on three challenging datasets demonstrate that the depth maps of input images enable our method to provide much more accurate alignment in the overlapping region and view-consistent results in the non-overlapping region. We believe our method will continue to work under the rapid progress of monocular depth estimation. The source code is available at https://github.com/tlliao/NIS_depths.