Multi-Scale Representation Learning for Image Restoration with State-Space Model

TL;DR

This paper introduces a multi-scale state-space model for image restoration that improves detail reconstruction and efficiency across various tasks by integrating novel modules and blocks, outperforming existing methods.

Contribution

The paper proposes a novel Multi-Scale State-Space Model with global and regional modules, along with AGB and RFB, to enhance multi-scale representation learning and detail extraction in image restoration.

Findings

Achieves state-of-the-art performance on nine benchmarks

Maintains low computational complexity

Effective across multiple image restoration tasks

Abstract

Image restoration endeavors to reconstruct a high-quality, detail-rich image from a degraded counterpart, which is a pivotal process in photography and various computer vision systems. In real-world scenarios, different types of degradation can cause the loss of image details at various scales and degrade image contrast. Existing methods predominantly rely on CNN and Transformer to capture multi-scale representations. However, these methods are often limited by the high computational complexity of Transformers and the constrained receptive field of CNN, which hinder them from achieving superior performance and efficiency in image restoration. To address these challenges, we propose a novel Multi-Scale State-Space Model-based (MS-Mamba) for efficient image restoration that enhances the capacity for multi-scale representation learning through our proposed global and regional SSM modules. Additionally, an Adaptive Gradient Block (AGB) and a Residual Fourier Block (RFB) are proposed to improve the network's detail extraction capabilities by capturing gradients in various directions and facilitating learning details in the frequency domain. Extensive experiments on nine public benchmarks across four classic image restoration tasks, image deraining, dehazing, denoising, and low-light enhancement, demonstrate that our proposed method achieves new state-of-the-art performance while maintaining low computational complexity. The source code will be publicly available.