Video Stitching for Linear Camera Arrays

TL;DR

This paper introduces a novel pushbroom stitching network for linear camera arrays that achieves temporally stable, high-quality video stitching with strong parallax tolerance, outperforming existing methods significantly.

Contribution

It presents a new deep learning-based approach using a pushbroom interpolation layer to learn dense flow fields for smooth spatial video stitching.

Findings

Outperforms state-of-the-art video stitching methods

Achieves high temporal stability and parallax tolerance

Validated through user study and practical applications

Abstract

Despite the long history of image and video stitching research, existing academic and commercial solutions still produce strong artifacts. In this work, we propose a wide-baseline video stitching algorithm for linear camera arrays that is temporally stable and tolerant to strong parallax. Our key insight is that stitching can be cast as a problem of learning a smooth spatial interpolation between the input videos. To solve this problem, inspired by pushbroom cameras, we introduce a fast pushbroom interpolation layer and propose a novel pushbroom stitching network, which learns a dense flow field to smoothly align the multiple input videos for spatial interpolation. Our approach outperforms the state-of-the-art by a significant margin, as we show with a user study, and has immediate applications in many areas such as virtual reality, immersive telepresence, autonomous driving, and video surveillance.