MambaIRv2: Attentive State Space Restoration

TL;DR

MambaIRv2 enhances image restoration by integrating non-causal attention into Mamba, enabling more effective pixel utilization and interaction, leading to superior performance with fewer parameters.

Contribution

It introduces a non-causal attentive state space model for Mamba, incorporating semantic-guided mechanisms to improve image restoration quality.

Findings

Outperforms SRFormer by 0.35dB PSNR with fewer parameters.

Surpasses HAT on classic SR by up to 0.29dB.

Achieves efficient image unfolding with a single scan.

Abstract

The Mamba-based image restoration backbones have recently demonstrated significant potential in balancing global reception and computational efficiency. However, the inherent causal modeling limitation of Mamba, where each token depends solely on its predecessors in the scanned sequence, restricts the full utilization of pixels across the image and thus presents new challenges in image restoration. In this work, we propose MambaIRv2, which equips Mamba with the non-causal modeling ability similar to ViTs to reach the attentive state space restoration model. Specifically, the proposed attentive state-space equation allows to attend beyond the scanned sequence and facilitate image unfolding with just one single scan. Moreover, we further introduce a semantic-guided neighboring mechanism to encourage interaction between distant but similar pixels. Extensive experiments show our MambaIRv2 outperforms SRFormer by even 0.35dB PSNR for lightweight SR even with 9.3\% less parameters and suppresses HAT on classic SR by up to 0.29dB. Code is available at https://github.com/csguoh/MambaIR.