Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes

TL;DR

This paper introduces a novel recurrent video deblurring method that employs blur-invariant motion estimation and pixel volumes to improve the aggregation of information from multiple frames, achieving state-of-the-art results.

Contribution

The paper proposes two new techniques: blur-invariant motion estimation learning and pixel volume-based motion compensation, enhancing deblurring accuracy for blurry videos.

Findings

Achieves state-of-the-art quantitative performance.

Outperforms recent deep learning-based methods.

Effectively exploits previous deblurred frames.

Abstract

For the success of video deblurring, it is essential to utilize information from neighboring frames. Most state-of-the-art video deblurring methods adopt motion compensation between video frames to aggregate information from multiple frames that can help deblur a target frame. However, the motion compensation methods adopted by previous deblurring methods are not blur-invariant, and consequently, their accuracy is limited for blurry frames with different blur amounts. To alleviate this problem, we propose two novel approaches to deblur videos by effectively aggregating information from multiple video frames. First, we present blur-invariant motion estimation learning to improve motion estimation accuracy between blurry frames. Second, for motion compensation, instead of aligning frames by warping with estimated motions, we use a pixel volume that contains candidate sharp pixels to resolve motion estimation errors. We combine these two processes to propose an effective recurrent video deblurring network that fully exploits deblurred previous frames. Experiments show that our method achieves the state-of-the-art performance both quantitatively and qualitatively compared to recent methods that use deep learning.