PlainUSR: Chasing Faster ConvNet for Efficient Super-Resolution

TL;DR

PlainUSR introduces a fast, efficient super-resolution framework that balances speed and quality by reparameterizing convolutional blocks, employing local importance-based attention, and using a plain U-Net backbone, achieving real-time performance.

Contribution

It proposes novel modifications to ConvNet components, including reparameterized convolutional blocks, local importance-based attention, and a simple U-Net backbone, to significantly reduce latency in super-resolution.

Findings

PlainUSR achieves 16.4x faster speed than NGswin with competitive quality.

The framework exhibits low latency and high scalability.

It outperforms existing latency-oriented super-resolution methods.

Abstract

Reducing latency is a roaring trend in recent super-resolution (SR) research. While recent progress exploits various convolutional blocks, attention modules, and backbones to unlock the full potentials of the convolutional neural network (ConvNet), achieving real-time performance remains a challenge. To this end, we present PlainUSR, a novel framework incorporating three pertinent modifications to expedite ConvNet for efficient SR. For the convolutional block, we squeeze the lighter but slower MobileNetv3 block into a heavier but faster vanilla convolution by reparameterization tricks to balance memory access and calculations. For the attention module, by modulating input with a regional importance map and gate, we introduce local importance-based attention to realize high-order information interaction within a 1-order attention latency. As to the backbone, we propose a plain U-Net that executes channel-wise discriminate splitting and concatenation. In the experimental phase, PlainUSR exhibits impressively low latency, great scalability, and competitive performance compared to both state-of-the-art latency-oriented and quality-oriented methods. In particular, compared to recent NGswin, the PlainUSR-L is 16.4x faster with competitive performance.