Learning Frequency-aware Dynamic Network for Efficient Super-Resolution

TL;DR

This paper introduces a frequency-aware dynamic network for single image super-resolution that divides input based on DCT coefficients, enabling significant computational savings while maintaining high image quality.

Contribution

It proposes a novel frequency-aware dynamic network that allocates computational resources adaptively based on DCT domain analysis for efficient super-resolution.

Findings

Reduces FLOPs by approximately 50%

Maintains state-of-the-art super-resolution performance

Effective across various neural architectures

Abstract

Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better fidelity and visual quality, most of existing networks are of heavy design with massive computation. However, the computation resources of modern mobile devices are limited, which cannot easily support the expensive cost. To this end, this paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Since pixels or image patches belong to low-frequency areas contain relatively few textural details, this dynamic network will not affect the quality of resulting super-resolution images. In addition, we embed predictors into the proposed dynamic network to end-to-end fine-tune the handcrafted frequency-aware masks. Extensive experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures to obtain the better tradeoff between visual quality and computational complexity. For instance, we can reduce the FLOPs of SR models by approximate 50% while preserving state-of-the-art SISR performance.