ML-CrAIST: Multi-scale Low-high Frequency Information-based Cross black Attention with Image Super-resolving Transformer

TL;DR

ML-CrAIST introduces a multi-scale transformer architecture that effectively leverages low-high frequency information and dual spatial-channel attention to improve single-image super-resolution performance.

Contribution

It proposes a novel transformer-based super-resolution model utilizing multi-scale frequency information and cross-attention between low and high frequencies, enhancing detail reconstruction.

Findings

Outperforms state-of-the-art methods with a 0.15 dB gain on Manga109 ×4.

Effectively models spatial and channel interactions for better super-resolution.

Utilizes cross-attention to exploit correlations between low and high-frequency information.

Abstract

Recently, transformers have captured significant interest in the area of single-image super-resolution tasks, demonstrating substantial gains in performance. Current models heavily depend on the network's extensive ability to extract high-level semantic details from images while overlooking the effective utilization of multi-scale image details and intermediate information within the network. Furthermore, it has been observed that high-frequency areas in images present significant complexity for super-resolution compared to low-frequency areas. This work proposes a transformer-based super-resolution architecture called ML-CrAIST that addresses this gap by utilizing low-high frequency information in multiple scales. Unlike most of the previous work (either spatial or channel), we operate spatial and channel self-attention, which concurrently model pixel interaction from both spatial and channel dimensions, exploiting the inherent correlations across spatial and channel axis. Further, we devise a cross-attention block for super-resolution, which explores the correlations between low and high-frequency information. Quantitative and qualitative assessments indicate that our proposed ML-CrAIST surpasses state-of-the-art super-resolution methods (e.g., 0.15 dB gain @Manga109 $\times$4). Code is available on: https://github.com/Alik033/ML-CrAIST.