Image-specific Convolutional Kernel Modulation for Single Image Super-resolution

TL;DR

This paper introduces an image-specific convolutional kernel modulation (IKM) technique for super-resolution that adaptively adjusts kernels based on image content, combined with an image-specific optimization method, enhancing performance without extra parameters.

Contribution

The paper proposes a novel IKM method with an image-specific optimization algorithm and a new U-Hourglass Dense Network architecture for improved super-resolution performance.

Findings

IKM outperforms existing attention mechanisms in super-resolution tasks.

The IsO algorithm is more effective than traditional mini-batch SGD.

The U-HDN architecture enhances the effectiveness of IKM experimentally.

Abstract

Recently, deep-learning-based super-resolution methods have achieved excellent performances, but mainly focus on training a single generalized deep network by feeding numerous samples. Yet intuitively, each image has its representation, and is expected to acquire an adaptive model. For this issue, we propose a novel image-specific convolutional kernel modulation (IKM) by exploiting the global contextual information of image or feature to generate an attention weight for adaptively modulating the convolutional kernels, which outperforms the vanilla convolution and several existing attention mechanisms while embedding into the state-of-the-art architectures without any additional parameters. Particularly, to optimize our IKM in mini-batch training, we introduce an image-specific optimization (IsO) algorithm, which is more effective than the conventional mini-batch SGD optimization. Furthermore, we investigate the effect of IKM on the state-of-the-art architectures and exploit a new backbone with U-style residual learning and hourglass dense block learning, terms U-Hourglass Dense Network (U-HDN), which is an appropriate architecture to utmost improve the effectiveness of IKM theoretically and experimentally. Extensive experiments on single image super-resolution show that the proposed methods achieve superior performances over state-of-the-art methods. Code is available at github.com/YuanfeiHuang/IKM.