QMamba: On First Exploration of Vision Mamba for Image Quality Assessment

TL;DR

This paper introduces QMamba, a novel vision foundation model adapted for image quality assessment, demonstrating superior perception and transferability across diverse IQA tasks with lower computational costs.

Contribution

It pioneers the adaptation of the Mamba foundation model for IQA, proposing StylePrompt tuning for enhanced transferability and efficiency in various IQA scenarios.

Findings

QMamba outperforms existing models like Swin Transformer, ViT, and CNNs in perception and efficiency.

StylePrompt tuning improves transferability with lower computational cost.

Extensive experiments validate QMamba's effectiveness across multiple IQA datasets.

Abstract

In this work, we take the first exploration of the recently popular foundation model, i.e., State Space Model/Mamba, in image quality assessment (IQA), aiming at observing and excavating the perception potential in vision Mamba. A series of works on Mamba has shown its significant potential in various fields, e.g., segmentation and classification. However, the perception capability of Mamba remains under-explored. Consequently, we propose QMamba by revisiting and adapting the Mamba model for three crucial IQA tasks, i.e., task-specific, universal, and transferable IQA, which reveals its clear advantages over existing foundational models, e.g., Swin Transformer, ViT, and CNNs, in terms of perception and computational cost. To improve the transferability of QMamba, we propose the StylePrompt tuning paradigm, where lightweight mean and variance prompts are injected to assist task-adaptive transfer learning of pre-trained QMamba for different downstream IQA tasks. Compared with existing prompt tuning strategies, our StylePrompt enables better perceptual transfer with lower computational cost. Extensive experiments on multiple synthetic, authentic IQA datasets, and cross IQA datasets demonstrate the effectiveness of our proposed QMamba. The code will be available at: https://github.com/bingo-G/QMamba.git