Efficient Star Distillation Attention Network for Lightweight Image Super-Resolution

TL;DR

This paper introduces an efficient lightweight image super-resolution network that combines novel attention and distillation modules to improve reconstruction quality while maintaining low computational costs.

Contribution

It proposes the Star Distillation Module and Multi-shape Multi-scale Large Kernel Attention to enhance feature learning and long-range dependency capture in lightweight CNNs.

Findings

Outperforms existing lightweight SISR methods in accuracy.

Achieves higher image quality with lower computational complexity.

Demonstrates superior visual results in super-resolution tasks.

Abstract

In recent years, the performance of lightweight Single-Image Super-Resolution (SISR) has been improved significantly with the application of Convolutional Neural Networks (CNNs) and Large Kernel Attention (LKA). However, existing information distillation modules for lightweight SISR struggle to map inputs into High-Dimensional Non-Linear (HDNL) feature spaces, limiting their representation learning. And their LKA modules possess restricted ability to capture the multi-shape multi-scale information for long-range dependencies while encountering a quadratic increase in the computational burden with increasing convolutional kernel size of its depth-wise convolutional layer. To address these issues, we firstly propose a Star Distillation Module (SDM) to enhance the discriminative representation learning via information distillation in the HDNL feature spaces. Besides, we present a Multi-shape Multi-scale Large Kernel Attention (MM-LKA) module to learn representative long-range dependencies while incurring low computational and memory footprints, leading to improving the performance of CNN-based self-attention significantly. Integrating SDM and MM-LKA, we develop a Residual Star Distillation Attention Module (RSDAM) and take it as the building block of the proposed efficient Star Distillation Attention Network (SDAN) which possesses high reconstruction efficiency to recover a higher-quality image from the corresponding low-resolution (LR) counterpart. When compared with other lightweight state-of-the-art SISR methods, extensive experiments show that our SDAN with low model complexity yields superior performance quantitatively and visually.