MS-SCANet: A Multiscale Transformer-Based Architecture with Dual Attention for No-Reference Image Quality Assessment

TL;DR

MS-SCANet is a multiscale transformer architecture with dual attention mechanisms designed for no-reference image quality assessment, effectively capturing details across scales and outperforming existing methods.

Contribution

Introduces a novel multiscale transformer model with dual attention and new consistency losses for improved no-reference IQA performance.

Findings

Outperforms state-of-the-art IQA methods on multiple datasets.

Effectively captures both fine and coarse image details.

Maintains spatial integrity with new consistency loss functions.

Abstract

We present the Multi-Scale Spatial Channel Attention Network (MS-SCANet), a transformer-based architecture designed for no-reference image quality assessment (IQA). MS-SCANet features a dual-branch structure that processes images at multiple scales, effectively capturing both fine and coarse details, an improvement over traditional single-scale methods. By integrating tailored spatial and channel attention mechanisms, our model emphasizes essential features while minimizing computational complexity. A key component of MS-SCANet is its cross-branch attention mechanism, which enhances the integration of features across different scales, addressing limitations in previous approaches. We also introduce two new consistency loss functions, Cross-Branch Consistency Loss and Adaptive Pooling Consistency Loss, which maintain spatial integrity during feature scaling, outperforming conventional linear and bilinear techniques. Extensive evaluations on datasets like KonIQ-10k, LIVE, LIVE Challenge, and CSIQ show that MS-SCANet consistently surpasses state-of-the-art methods, offering a robust framework with stronger correlations with subjective human scores.