Transforming Image Super-Resolution: A ConvFormer-based Efficient Approach

TL;DR

This paper introduces ConvFormer, a novel lightweight super-resolution network that combines convolution and transformer techniques to achieve high performance with reduced computational costs, suitable for resource-limited devices.

Contribution

The paper proposes ConvFormer-based Super-Resolution (CFSR), a new efficient architecture that replaces self-attention with large kernel convolutions and includes an edge-preserving network for high-frequency detail retention.

Findings

CFSR outperforms existing lightweight SR methods in accuracy and efficiency.

Achieves 0.39 dB higher PSNR on Urban100 dataset for x2 super-resolution.

Reduces parameters by 26% and FLOPs by 31% compared to state-of-the-art methods.

Abstract

Recent progress in single-image super-resolution (SISR) has achieved remarkable performance, yet the computational costs of these methods remain a challenge for deployment on resource-constrained devices. In particular, transformer-based methods, which leverage self-attention mechanisms, have led to significant breakthroughs but also introduce substantial computational costs. To tackle this issue, we introduce the Convolutional Transformer layer (ConvFormer) and propose a ConvFormer-based Super-Resolution network (CFSR), offering an effective and efficient solution for lightweight image super-resolution. The proposed method inherits the advantages of both convolution-based and transformer-based approaches. Specifically, CFSR utilizes large kernel convolutions as a feature mixer to replace the self-attention module, efficiently modeling long-range dependencies and extensive receptive fields with minimal computational overhead. Furthermore, we propose an edge-preserving feed-forward network (EFN) designed to achieve local feature aggregation while effectively preserving high-frequency information. Extensive experiments demonstrate that CFSR strikes an optimal balance between computational cost and performance compared to existing lightweight SR methods. When benchmarked against state-of-the-art methods such as ShuffleMixer, the proposed CFSR achieves a gain of 0.39 dB on the Urban100 dataset for the x2 super-resolution task while requiring 26\% and 31\% fewer parameters and FLOPs, respectively. The code and pre-trained models are available at https://github.com/Aitical/CFSR.