Fast Full-frame Video Stabilization with Iterative Optimization

TL;DR

This paper introduces a fast, iterative optimization method for full-frame video stabilization that balances visual quality and computational efficiency through a novel divide-and-conquer approach and fixed-point theory.

Contribution

It presents a new iterative optimization framework with a multiframe fusion strategy and probabilistic flow guidance, advancing the speed and quality of video stabilization.

Findings

Outperforms existing methods in speed and quality

Guarantees convergence using fixed-point theory

Effective in stabilizing shaky videos with high visual fidelity

Abstract

Video stabilization refers to the problem of transforming a shaky video into a visually pleasing one. The question of how to strike a good trade-off between visual quality and computational speed has remained one of the open challenges in video stabilization. Inspired by the analogy between wobbly frames and jigsaw puzzles, we propose an iterative optimization-based learning approach using synthetic datasets for video stabilization, which consists of two interacting submodules: motion trajectory smoothing and full-frame outpainting. First, we develop a two-level (coarse-to-fine) stabilizing algorithm based on the probabilistic flow field. The confidence map associated with the estimated optical flow is exploited to guide the search for shared regions through backpropagation. Second, we take a divide-and-conquer approach and propose a novel multiframe fusion strategy to render full-frame stabilized views. An important new insight brought about by our iterative optimization approach is that the target video can be interpreted as the fixed point of nonlinear mapping for video stabilization. We formulate video stabilization as a problem of minimizing the amount of jerkiness in motion trajectories, which guarantees convergence with the help of fixed-point theory. Extensive experimental results are reported to demonstrate the superiority of the proposed approach in terms of computational speed and visual quality. The code will be available on GitHub.