DC-VSR: Spatially and Temporally Consistent Video Super-Resolution with Video Diffusion Prior

TL;DR

DC-VSR introduces a novel diffusion-based video super-resolution method that ensures spatial and temporal consistency by leveraging attention mechanisms and a new guidance scheme, resulting in high-quality, realistic videos.

Contribution

The paper proposes DC-VSR, a new approach that combines spatial and temporal attention propagation with a diffusion guidance scheme to improve video super-resolution quality and consistency.

Findings

Outperforms previous VSR methods in quality and consistency.

Achieves realistic textures with spatially and temporally coherent details.

Demonstrates effectiveness through comprehensive experiments.

Abstract

Video super-resolution (VSR) aims to reconstruct a high-resolution (HR) video from a low-resolution (LR) counterpart. Achieving successful VSR requires producing realistic HR details and ensuring both spatial and temporal consistency. To restore realistic details, diffusion-based VSR approaches have recently been proposed. However, the inherent randomness of diffusion, combined with their tile-based approach, often leads to spatio-temporal inconsistencies. In this paper, we propose DC-VSR, a novel VSR approach to produce spatially and temporally consistent VSR results with realistic textures. To achieve spatial and temporal consistency, DC-VSR adopts a novel Spatial Attention Propagation (SAP) scheme and a Temporal Attention Propagation (TAP) scheme that propagate information across spatio-temporal tiles based on the self-attention mechanism. To enhance high-frequency details, we also introduce Detail-Suppression Self-Attention Guidance (DSSAG), a novel diffusion guidance scheme. Comprehensive experiments demonstrate that DC-VSR achieves spatially and temporally consistent, high-quality VSR results, outperforming previous approaches.