Rethinking Diffusion Model-Based Video Super-Resolution: Leveraging Dense Guidance from Aligned Features

TL;DR

This paper introduces DGAF-VSR, a novel diffusion model-based video super-resolution method that leverages dense feature guidance and aligned features to improve perceptual quality, fidelity, and temporal consistency.

Contribution

It proposes a new framework with an optical guided warping module and a feature-wise temporal condition module to better align and compensate video frames in the feature domain.

Findings

DGAF-VSR achieves 35.82% DISTS reduction, indicating improved perceptual quality.

It attains 0.20 dB PSNR gain, showing enhanced fidelity.

The method reduces tLPIPS by 30.37%, improving temporal consistency.

Abstract

Diffusion model (DM) based Video Super-Resolution (VSR) approaches achieve impressive perceptual quality. However, they suffer from error accumulation, spatial artifacts, and a trade-off between perceptual quality and fidelity, primarily caused by inaccurate alignment and insufficient compensation between video frames. In this paper, within the DM-based VSR pipeline, we revisit the role of alignment and compensation between adjacent video frames and reveal two crucial observations: (a) the feature domain is better suited than the pixel domain for information compensation due to its stronger spatial and temporal correlations, and (b) warping at an upscaled resolution better preserves high-frequency information, but this benefit is not necessarily monotonic. Therefore, we propose a novel Densely Guided diffusion model with Aligned Features for Video Super-Resolution (DGAF-VSR), with an Optical Guided Warping Module (OGWM) to maintain high-frequency details in the aligned features and a Feature-wise Temporal Condition Module (FTCM) to deliver dense guidance in the feature domain. Extensive experiments on synthetic and real-world datasets demonstrate that DGAF-VSR surpasses state-of-the-art methods in key aspects of VSR, including perceptual quality (35.82\% DISTS reduction), fidelity (0.20 dB PSNR gain), and temporal consistency (30.37\% tLPIPS reduction).