Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restoration

TL;DR

This paper introduces TAMambaIR, a texture-aware state space model that improves image restoration by focusing on complex textures, balancing quality and efficiency across various tasks like super-resolution and deraining.

Contribution

The paper proposes a novel texture-aware state space model and multi-directional perception block, enhancing texture modeling and computational efficiency in image restoration.

Findings

Achieves state-of-the-art results on multiple image restoration benchmarks.

Significantly improves efficiency compared to existing CNN and Transformer-based methods.

Effectively models long-range dependencies and spatial texture characteristics.

Abstract

Image restoration aims to recover details and enhance contrast in degraded images. With the growing demand for high-quality imaging (\textit{e.g.}, 4K and 8K), achieving a balance between restoration quality and computational efficiency has become increasingly critical. Existing methods, primarily based on CNNs, Transformers, or their hybrid approaches, apply uniform deep representation extraction across the image. However, these methods often struggle to effectively model long-range dependencies and largely overlook the spatial characteristics of image degradation (regions with richer textures tend to suffer more severe damage), making it hard to achieve the best trade-off between restoration quality and efficiency. To address these issues, we propose a novel texture-aware image restoration method, TAMambaIR, which simultaneously perceives image textures and achieves a trade-off between performance and efficiency. Specifically, we introduce a novel Texture-Aware State Space Model, which enhances texture awareness and improves efficiency by modulating the transition matrix of the state-space equation and focusing on regions with complex textures. Additionally, we design a {Multi-Directional Perception Block} to improve multi-directional receptive fields while maintaining low computational overhead. Extensive experiments on benchmarks for image super-resolution, deraining, and low-light image enhancement demonstrate that TAMambaIR achieves state-of-the-art performance with significantly improved efficiency, establishing it as a robust and efficient framework for image restoration.