Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion

TL;DR

This paper introduces OSDEnhancer, a novel one-step diffusion framework for space-time video super-resolution that effectively handles real-world complex degradations and enhances both spatial and temporal details.

Contribution

The paper presents the first one-step diffusion-based STVSR method with a divide-and-conquer strategy and specialized LoRAs for improved real-world performance.

Findings

Achieves state-of-the-art results in real-world scenarios.

Demonstrates superior generalization over existing methods.

Effectively recovers fine textures and maintains temporal coherence.

Abstract

Diffusion models have demonstrated exceptional success in video super-resolution (VSR), exhibiting powerful capabilities for generating fine-grained details. However, their potential for space-time video super-resolution (STVSR), which necessitates not only recovering realistic high-resolution visual content but also improving the frame rate with coherent temporal dynamics, remains largely underexplored. Moreover, existing STVSR methods predominantly address spatiotemporal upsampling under simple degradation assumptions, thus failing in real-world scenarios with complex unknown degradations. To address these challenges, we propose OSDEnhancer, the first framework that achieves robust STVSR in one-step diffusion. OSDEnhancer begins with a linear initialization to establish essential spatiotemporal structures and adapt the model for one-step reconstruction. It then applies a divide-and-conquer strategy, introducing the temporal coherence (TC) and texture enrichment (TE) LoRAs that progressively specialize in inter-frame dynamics modeling and fine-grained texture recovery, respectively, while collaborating during inference for enhanced overall performance. A bidirectional VAE decoder employs deformable recurrent blocks to leverage the multi-scale structure of the vanilla VAE, enhancing latent-to-pixel reconstruction through joint multi-scale deformable aggregation and inter-frame feature propagation. Experimental results demonstrate that the proposed method attains state-of-the-art performance with superior generalization in real-world scenarios. The code is available at https://github.com/W-Shuoyan/OSDEnhancer.