$\text{S}^{3}$Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Model

TL;DR

S^{3}Mamba introduces a scalable state space model with a scale-aware self-attention mechanism to enable efficient and high-quality arbitrary-scale image super-resolution, overcoming limitations of existing CNN and transformer-based methods.

Contribution

It proposes a novel scalable state space model and a scale-aware self-attention mechanism for arbitrary-scale super-resolution, achieving linear computational complexity and improved global feature perception.

Findings

Achieves state-of-the-art performance on synthetic and real-world benchmarks.

Demonstrates superior generalization across arbitrary scales.

Outperforms existing methods in quality and efficiency.

Abstract

Arbitrary scale super-resolution (ASSR) aims to super-resolve low-resolution images to high-resolution images at any scale using a single model, addressing the limitations of traditional super-resolution methods that are restricted to fixed-scale factors (e.g., $\times2$, $\times4$). The advent of Implicit Neural Representations (INR) has brought forth a plethora of novel methodologies for ASSR, which facilitate the reconstruction of original continuous signals by modeling a continuous representation space for coordinates and pixel values, thereby enabling arbitrary-scale super-resolution. Consequently, the primary objective of ASSR is to construct a continuous representation space derived from low-resolution inputs. However, existing methods, primarily based on CNNs and Transformers, face significant challenges such as high computational complexity and inadequate modeling of long-range dependencies, which hinder their effectiveness in real-world applications. To overcome these limitations, we propose a novel arbitrary-scale super-resolution method, called $\text{S}^{3}$Mamba, to construct a scalable continuous representation space. Specifically, we propose a Scalable State Space Model (SSSM) to modulate the state transition matrix and the sampling matrix of step size during the discretization process, achieving scalable and continuous representation modeling with linear computational complexity. Additionally, we propose a novel scale-aware self-attention mechanism to further enhance the network's ability to perceive global important features at different scales, thereby building the $\text{S}^{3}$Mamba to achieve superior arbitrary-scale super-resolution. Extensive experiments on both synthetic and real-world benchmarks demonstrate that our method achieves state-of-the-art performance and superior generalization capabilities at arbitrary super-resolution scales.