MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMs

TL;DR

MambaCSR introduces a dual-interleaved scanning framework for compressed image super-resolution, effectively capturing multi-scale contextual information while reducing computational costs, leading to superior performance on benchmarks.

Contribution

The paper proposes a novel dual-interleaved scanning paradigm for CSR, combining hierarchical and horizontal-vertical strategies to enhance contextual modeling and efficiency.

Findings

Outperforms existing methods on multiple benchmarks

Effectively models multi-scale contextual information

Reduces computational cost with interleaved scanning

Abstract

We present MambaCSR, a simple but effective framework based on Mamba for the challenging compressed image super-resolution (CSR) task. Particularly, the scanning strategies of Mamba are crucial for effective contextual knowledge modeling in the restoration process despite it relying on selective state space modeling for all tokens. In this work, we propose an efficient dual-interleaved scanning paradigm (DIS) for CSR, which is composed of two scanning strategies: (i) hierarchical interleaved scanning is designed to comprehensively capture and utilize the most potential contextual information within an image by simultaneously taking advantage of the local window-based and sequential scanning methods; (ii) horizontal-to-vertical interleaved scanning is proposed to reduce the computational cost by leaving the redundancy between the scanning of different directions. To overcome the non-uniform compression artifacts, we also propose position-aligned cross-scale scanning to model multi-scale contextual information. Experimental results on multiple benchmarks have shown the great performance of our MambaCSR in the compressed image super-resolution task. The code will be soon available in~\textcolor{magenta}{\url{https://github.com/renyulin-f/MambaCSR}}.