VSRM: A Robust Mamba-Based Framework for Video Super-Resolution

TL;DR

VSRM introduces a Mamba-based framework for video super-resolution that efficiently models long-range spatio-temporal features, improves frame alignment, and enhances high-frequency detail preservation, achieving state-of-the-art results.

Contribution

The paper presents a novel Mamba-based framework with specialized modules for long-range feature extraction and flexible frame alignment in video super-resolution.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Effectively models long-range spatio-temporal dependencies.

Preserves high-frequency details with a new frequency domain loss.

Abstract

Video super-resolution remains a major challenge in low-level vision tasks. To date, CNN- and Transformer-based methods have delivered impressive results. However, CNNs are limited by local receptive fields, while Transformers struggle with quadratic complexity, posing challenges for processing long sequences in VSR. Recently, Mamba has drawn attention for its long-sequence modeling, linear complexity, and large receptive fields. In this work, we propose VSRM, a novel \textbf{V}ideo \textbf{S}uper-\textbf{R}esolution framework that leverages the power of \textbf{M}amba. VSRM introduces Spatial-to-Temporal Mamba and Temporal-to-Spatial Mamba blocks to extract long-range spatio-temporal features and enhance receptive fields efficiently. To better align adjacent frames, we propose Deformable Cross-Mamba Alignment module. This module utilizes a deformable cross-mamba mechanism to make the compensation stage more dynamic and flexible, preventing feature distortions. Finally, we minimize the frequency domain gaps between reconstructed and ground-truth frames by proposing a simple yet effective Frequency Charbonnier-like loss that better preserves high-frequency content and enhances visual quality. Through extensive experiments, VSRM achieves state-of-the-art results on diverse benchmarks, establishing itself as a solid foundation for future research.