Weakly-Supervised Stitching Network for Real-World Panoramic Image Generation

TL;DR

This paper introduces a weakly-supervised deep learning model for stitching multiple fisheye images into a seamless 360-degree panoramic image, overcoming the need for ground truth data.

Contribution

It proposes a novel weakly-supervised training mechanism and a stitching model that handles real-world fisheye images for panoramic generation.

Findings

Effective on real-world datasets

Produces high-quality 360-degree images

Does not require ground truth panoramic images

Abstract

Recently, there has been growing attention on an end-to-end deep learning-based stitching model. However, the most challenging point in deep learning-based stitching is to obtain pairs of input images with a narrow field of view and ground truth images with a wide field of view captured from real-world scenes. To overcome this difficulty, we develop a weakly-supervised learning mechanism to train the stitching model without requiring genuine ground truth images. In addition, we propose a stitching model that takes multiple real-world fisheye images as inputs and creates a 360 output image in an equirectangular projection format. In particular, our model consists of color consistency corrections, warping, and blending, and is trained by perceptual and SSIM losses. The effectiveness of the proposed algorithm is verified on two real-world stitching datasets.