Implicit Grid Convolution for Multi-Scale Image Super-Resolution

TL;DR

This paper introduces Implicit Grid Convolution, a unified upsampling method for multi-scale image super-resolution that reduces training time and model size while maintaining high performance.

Contribution

It proposes a novel Implicit Grid Convolution (IGConv) that unifies scale-specific upsamplers, enabling efficient multi-scale training with comparable results to traditional methods.

Findings

Achieves similar super-resolution performance with three times fewer parameters.

Reduces training time and storage requirements significantly.

Improves PSNR by 0.21dB on Urban100×4 with IGConv+.

Abstract

For Image Super-Resolution (SR), it is common to train and evaluate scale-specific models composed of an encoder and upsampler for each targeted scale. Consequently, many SR studies encounter substantial training times and complex deployment requirements. In this paper, we address this limitation by training and evaluating multiple scales simultaneously. Notably, we observe that encoder features are similar across scales and that the Sub-Pixel Convolution (SPConv), widely-used scale-specific upsampler, exhibits strong inter-scale correlations in its functionality. Building on these insights, we propose a multi-scale framework that employs a single encoder in conjunction with Implicit Grid Convolution (IGConv), our novel upsampler, which unifies SPConv across all scales within a single module. Extensive experiments demonstrate that our framework achieves comparable performance to existing fixed-scale methods while reducing the training budget and stored parameters three-fold and maintaining the same latency. Additionally, we propose IGConv$^{+}$ to improve performance further by addressing spectral bias and allowing input-dependent upsampling and ensembled prediction. As a result, ATD-IGConv$^{+}$ achieves a notable 0.21dB improvement in PSNR on Urban100$\times$4, while also reducing the training budget, stored parameters, and inference cost compared to the existing ATD.