Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution

TL;DR

This paper introduces ASID, a lightweight Transformer-based super-resolution network that reduces computational complexity through attention-sharing and information distillation, achieving competitive performance with fewer parameters.

Contribution

The paper proposes a novel lightweight SR network, ASID, that incorporates attention-sharing and an adapted information distillation scheme for Transformer-based methods.

Findings

ASID requires only around 300K parameters.

ASID outperforms state-of-the-art SR methods with similar parameter counts.

ASID achieves competitive performance with significantly fewer parameters.

Abstract

Transformer-based Super-Resolution (SR) methods have demonstrated superior performance compared to convolutional neural network (CNN)-based SR approaches due to their capability to capture long-range dependencies. However, their high computational complexity necessitates the development of lightweight approaches for practical use. To address this challenge, we propose the Attention-Sharing Information Distillation (ASID) network, a lightweight SR network that integrates attention-sharing and an information distillation structure specifically designed for Transformer-based SR methods. We modify the information distillation scheme, originally designed for efficient CNN operations, to reduce the computational load of stacked self-attention layers, effectively addressing the efficiency bottleneck. Additionally, we introduce attention-sharing across blocks to further minimize the computational cost of self-attention operations. By combining these strategies, ASID achieves competitive performance with existing SR methods while requiring only around 300K parameters - significantly fewer than existing CNN-based and Transformer-based SR models. Furthermore, ASID outperforms state-of-the-art SR methods when the number of parameters is matched, demonstrating its efficiency and effectiveness. The code and supplementary material are available on the project page.