Partial Large Kernel CNNs for Efficient Super-Resolution

TL;DR

This paper introduces Partial Large Kernel CNNs for efficient super-resolution, combining large kernels with novel techniques to significantly reduce latency and memory usage while achieving state-of-the-art results.

Contribution

It proposes a new CNN architecture that incorporates large kernels and element-wise attention to improve efficiency and performance in super-resolution tasks.

Findings

Achieves state-of-the-art performance on four datasets at 4x scale.

Reduces latency by 86% compared to naive large kernel approaches.

Decreases maximum GPU memory occupancy by 80.2%.

Abstract

Recently, in the super-resolution (SR) domain, transformers have outperformed CNNs with fewer FLOPs and fewer parameters since they can deal with long-range dependency and adaptively adjust weights based on instance. In this paper, we demonstrate that CNNs, although less focused on in the current SR domain, surpass Transformers in direct efficiency measures. By incorporating the advantages of Transformers into CNNs, we aim to achieve both computational efficiency and enhanced performance. However, using a large kernel in the SR domain, which mainly processes large images, incurs a large computational overhead. To overcome this, we propose novel approaches to employing the large kernel, which can reduce latency by 86\% compared to the naive large kernel, and leverage an Element-wise Attention module to imitate instance-dependent weights. As a result, we introduce Partial Large Kernel CNNs for Efficient Super-Resolution (PLKSR), which achieves state-of-the-art performance on four datasets at a scale of $\times$4, with reductions of 68.1\% in latency and 80.2\% in maximum GPU memory occupancy compared to SRFormer-light.