Joint Flow And Feature Refinement Using Attention For Video Restoration

TL;DR

This paper introduces JFFRA, a novel video restoration framework that iteratively refines flow and features using attention mechanisms, improving temporal consistency and performance across various tasks.

Contribution

The paper proposes a new joint flow and feature refinement method that reduces reliance on precise flow estimation and enhances restoration quality through iterative collaboration.

Findings

Achieves up to 1.62 dB improvement over state-of-the-art methods.

Effectively reduces flickering artifacts with occlusion-aware loss.

Demonstrates versatility across denoising, deblurring, and super-resolution.

Abstract

Recent advancements in video restoration have focused on recovering high-quality video frames from low-quality inputs. Compared with static images, the performance of video restoration significantly depends on efficient exploitation of temporal correlations among successive video frames. The numerous techniques make use of temporal information via flow-based strategies or recurrent architectures. However, these methods often encounter difficulties in preserving temporal consistency as they utilize degraded input video frames. To resolve this issue, we propose a novel video restoration framework named Joint Flow and Feature Refinement using Attention (JFFRA). The proposed JFFRA is based on key philosophy of iteratively enhancing data through the synergistic collaboration of flow (alignment) and restoration. By leveraging previously enhanced features to refine flow and vice versa, JFFRA enables efficient feature enhancement using temporal information. This interplay between flow and restoration is executed at multiple scales, reducing the dependence on precise flow estimation. Moreover, we incorporate an occlusion-aware temporal loss function to enhance the network's capability in eliminating flickering artifacts. Comprehensive experiments validate the versatility of JFFRA across various restoration tasks such as denoising, deblurring, and super-resolution. Our method demonstrates a remarkable performance improvement of up to 1.62 dB compared to state-of-the-art approaches.