Spatio-Temporal Deformable Attention Network for Video Deblurring

TL;DR

This paper introduces STDANet, a novel video deblurring network that adaptively focuses on sharp pixels by considering pixel-wise blur levels, improving restoration quality over existing methods.

Contribution

The paper proposes the spatio-temporal deformable attention network (STDANet) that leverages pixel-wise blur levels and deformable attention for enhanced video deblurring.

Findings

STDANet outperforms state-of-the-art methods on multiple datasets.

It effectively extracts sharp pixels by considering pixel-wise blur levels.

The approach improves deblurring quality in challenging scenarios.

Abstract

The key success factor of the video deblurring methods is to compensate for the blurry pixels of the mid-frame with the sharp pixels of the adjacent video frames. Therefore, mainstream methods align the adjacent frames based on the estimated optical flows and fuse the alignment frames for restoration. However, these methods sometimes generate unsatisfactory results because they rarely consider the blur levels of pixels, which may introduce blurry pixels from video frames. Actually, not all the pixels in the video frames are sharp and beneficial for deblurring. To address this problem, we propose the spatio-temporal deformable attention network (STDANet) for video delurring, which extracts the information of sharp pixels by considering the pixel-wise blur levels of the video frames. Specifically, STDANet is an encoder-decoder network combined with the motion estimator and spatio-temporal deformable attention (STDA) module, where motion estimator predicts coarse optical flows that are used as base offsets to find the corresponding sharp pixels in STDA module. Experimental results indicate that the proposed STDANet performs favorably against state-of-the-art methods on the GoPro, DVD, and BSD datasets.