Three-Stage Cascade Framework for Blurry Video Frame Interpolation

TL;DR

This paper introduces a three-stage cascade framework for blurry video frame interpolation that effectively leverages information in blurry videos to generate high-quality, high-frame-rate clear videos, outperforming existing methods.

Contribution

The paper proposes a novel end-to-end three-stage framework combining deformable sampling, long-term temporal fusion, and transformer-based deblurring for BVFI.

Findings

Outperforms state-of-the-art methods on multiple benchmarks.

Demonstrates strong generalization on real-world blurry videos.

Effectively leverages information in blurry frames for high-quality interpolation.

Abstract

Blurry video frame interpolation (BVFI) aims to generate high-frame-rate clear videos from low-frame-rate blurry videos, is a challenging but important topic in the computer vision community. Blurry videos not only provide spatial and temporal information like clear videos, but also contain additional motion information hidden in each blurry frame. However, existing BVFI methods usually fail to fully leverage all valuable information, which ultimately hinders their performance. In this paper, we propose a simple end-to-end three-stage framework to fully explore useful information from blurry videos. The frame interpolation stage designs a temporal deformable network to directly sample useful information from blurry inputs and synthesize an intermediate frame at an arbitrary time interval. The temporal feature fusion stage explores the long-term temporal information for each target frame through a bi-directional recurrent deformable alignment network. And the deblurring stage applies a transformer-empowered Taylor approximation network to recursively recover the high-frequency details. The proposed three-stage framework has clear task assignment for each module and offers good expandability, the effectiveness of which are demonstrated by various experimental results. We evaluate our model on four benchmarks, including the Adobe240 dataset, GoPro dataset, YouTube240 dataset and Sony dataset. Quantitative and qualitative results indicate that our model outperforms existing SOTA methods. Besides, experiments on real-world blurry videos also indicate the good generalization ability of our model.