Enhancing Frequency for Single Image Super-Resolution with Learnable Separable Kernels

TL;DR

This paper introduces Learnable Separable Kernels (LSKs), a plug-and-play module that directly enhances image frequency components in single-image super-resolution, significantly reducing parameters and computational costs while improving performance.

Contribution

The paper proposes LSKs, a novel rank-one matrix-based module for SISR that directly targets frequency enhancement, with a decomposition for efficiency and interpretability.

Findings

LSKs reduce model parameters by over 60%.

LSKs improve super-resolution performance, especially at higher upscaling factors.

Visualization shows LSKs effectively enhance image frequency components.

Abstract

Existing approaches often enhance the performance of single-image super-resolution (SISR) methods by incorporating auxiliary structures, such as specialized loss functions, to indirectly boost the quality of low-resolution images. In this paper, we propose a plug-and-play module called Learnable Separable Kernels (LSKs), which are formally rank-one matrices designed to directly enhance image frequency components. We begin by explaining why LSKs are particularly suitable for SISR tasks from a frequency perspective. Baseline methods incorporating LSKs demonstrate a significant reduction of over 60\% in both the number of parameters and computational requirements. This reduction is achieved through the decomposition of LSKs into orthogonal and mergeable one-dimensional kernels. Additionally, we perform an interpretable analysis of the feature maps generated by LSKs. Visualization results reveal the capability of LSKs to enhance image frequency components effectively. Extensive experiments show that incorporating LSKs not only reduces the number of parameters and computational load but also improves overall model performance. Moreover, these experiments demonstrate that models utilizing LSKs exhibit superior performance, particularly as the upscaling factor increases.