GPU Accelerated Color Correction and Frame Warping for Real-time Video Stitching

TL;DR

This paper presents a GPU-accelerated real-time system for video stitching that maintains temporal consistency and color accuracy without requiring precise camera calibration, enabling high-quality panoramic videos.

Contribution

It introduces a novel real-time video stitching system using extended color correction methods and optical flow-based warping that does not depend on accurate camera parameters.

Findings

Achieves real-time panoramic video stitching with high quality.

Maintains temporal flicker and color consistency in videos.

Operates effectively without precise camera calibration.

Abstract

Traditional image stitching focuses on a single panorama frame without considering the spatial-temporal consistency in videos. The straightforward image stitching approach will cause temporal flicking and color inconstancy when it is applied to the video stitching task. Besides, inaccurate camera parameters will cause artifacts in the image warping. In this paper, we propose a real-time system to stitch multiple video sequences into a panoramic video, which is based on GPU accelerated color correction and frame warping without accurate camera parameters. We extend the traditional 2D-Matrix (2D-M) color correction approach and a present spatio-temporal 3D-Matrix (3D-M) color correction method for the overlap local regions with online color balancing using a piecewise function on global frames. Furthermore, we use pairwise homography matrices given by coarse camera calibration for global warping followed by accurate local warping based on the optical flow. Experimental results show that our system can generate highquality panorama videos in real time.