Pyramid Feature Alignment Network for Video Deblurring

TL;DR

The paper introduces PFAN, a novel pyramid feature alignment network that improves video deblurring by multi-scale feature extraction and cascade guided deformable alignment, effectively handling complex motions and misalignments.

Contribution

The paper presents a multi-scale feature extraction strategy with Structure-to-Detail Downsampling and a cascade guided deformable alignment for more accurate video deblurring.

Findings

PFAN outperforms state-of-the-art methods in accuracy.

The multi-scale approach effectively handles complex motions.

The method achieves competitive speed with superior performance.

Abstract

Video deblurring remains a challenging task due to various causes of blurring. Traditional methods have considered how to utilize neighboring frames by the single-scale alignment for restoration. However, they typically suffer from misalignment caused by severe blur. In this work, we aim to better utilize neighboring frames with high efficient feature alignment. We propose a Pyramid Feature Alignment Network (PFAN) for video deblurring. First, the multi-scale feature of blurry frames is extracted with the strategy of Structure-to-Detail Downsampling (SDD) before alignment. This downsampling strategy makes the edges sharper, which is helpful for alignment. Then we align the feature at each scale and reconstruct the image at the corresponding scale. This strategy effectively supervises the alignment at each scale, overcoming the problem of propagated errors from the above scales at the alignment stage. To better handle the challenges of complex and large motions, instead of aligning features at each scale separately, lower-scale motion information is used to guide the higher-scale motion estimation. Accordingly, a Cascade Guided Deformable Alignment (CGDA) is proposed to integrate coarse motion into deformable convolution for finer and more accurate alignment. As demonstrated in extensive experiments, our proposed PFAN achieves superior performance with competitive speed compared to the state-of-the-art methods.