Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring

TL;DR

This paper introduces a novel deep recurrent neural network with multi-scale bi-directional propagation for video deblurring, which effectively utilizes unaligned neighboring frames without explicit alignment estimation, and provides a new real-world blurry video dataset for evaluation.

Contribution

The paper proposes a simple yet effective RNN-based model with multi-scale bi-directional propagation that bypasses the need for explicit frame alignment in video deblurring.

Findings

The proposed dataset improves real-world video deblurring performance.

The RNN-MBP outperforms state-of-the-art methods on benchmark datasets.

The method effectively handles unaligned neighboring frames without explicit alignment.

Abstract

The success of the state-of-the-art video deblurring methods stems mainly from implicit or explicit estimation of alignment among the adjacent frames for latent video restoration. However, due to the influence of the blur effect, estimating the alignment information from the blurry adjacent frames is not a trivial task. Inaccurate estimations will interfere the following frame restoration. Instead of estimating alignment information, we propose a simple and effective deep Recurrent Neural Network with Multi-scale Bi-directional Propagation (RNN-MBP) to effectively propagate and gather the information from unaligned neighboring frames for better video deblurring. Specifically, we build a Multi-scale Bi-directional Propagation~(MBP) module with two U-Net RNN cells which can directly exploit the inter-frame information from unaligned neighboring hidden states by integrating them in different scales. Moreover, to better evaluate the proposed algorithm and existing state-of-the-art methods on real-world blurry scenes, we also create a Real-World Blurry Video Dataset (RBVD) by a well-designed Digital Video Acquisition System (DVAS) and use it as the training and evaluation dataset. Extensive experimental results demonstrate that the proposed RBVD dataset effectively improves the performance of existing algorithms on real-world blurry videos, and the proposed algorithm performs favorably against the state-of-the-art methods on three typical benchmarks. The code is available at https://github.com/XJTU-CVLAB-LOWLEVEL/RNN-MBP.