Semantic and Temporal Integration in Latent Diffusion Space for High-Fidelity Video Super-Resolution

TL;DR

SeTe-VSR introduces a novel latent diffusion-based method that combines semantic and temporal guidance to improve high-fidelity, temporally coherent video super-resolution, outperforming existing approaches.

Contribution

The paper presents a new approach integrating semantic and temporal guidance in latent diffusion space for enhanced video super-resolution.

Findings

Outperforms existing methods in detail recovery

Enhances perceptual quality of super-resolved videos

Maintains temporal coherence across frames

Abstract

Recent advancements in video super-resolution (VSR) models have demonstrated impressive results in enhancing low-resolution videos. However, due to limitations in adequately controlling the generation process, achieving high fidelity alignment with the low-resolution input while maintaining temporal consistency across frames remains a significant challenge. In this work, we propose Semantic and Temporal Guided Video Super-Resolution (SeTe-VSR), a novel approach that incorporates both semantic and temporal-spatio guidance in the latent diffusion space to address these challenges. By incorporating high-level semantic information and integrating spatial and temporal information, our approach achieves a seamless balance between recovering intricate details and ensuring temporal coherence. Our method not only preserves high-reality visual content but also significantly enhances fidelity. Extensive experiments demonstrate that SeTe-VSR outperforms existing methods in terms of detail recovery and perceptual quality, highlighting its effectiveness for complex video super-resolution tasks.