Wide Activation for Efficient and Accurate Image Super-Resolution

TL;DR

This paper introduces a wide activation approach in super-resolution networks, demonstrating improved accuracy and efficiency through wider feature maps, linear low-rank convolutions, and weight normalization, achieving state-of-the-art results.

Contribution

The paper proposes a novel wide activation design and low-rank convolution techniques for super-resolution, leading to better performance with less computational cost.

Findings

Wider features before ReLU improve super-resolution accuracy.

Linear low-rank convolutions enhance efficiency without sacrificing quality.

Weight normalization outperforms batch normalization in deep super-resolution models.

Abstract

In this report we demonstrate that with same parameters and computational budgets, models with wider features before ReLU activation have significantly better performance for single image super-resolution (SISR). The resulted SR residual network has a slim identity mapping pathway with wider (\(2\times\) to \(4\times\)) channels before activation in each residual block. To further widen activation (\(6\times\) to \(9\times\)) without computational overhead, we introduce linear low-rank convolution into SR networks and achieve even better accuracy-efficiency tradeoffs. In addition, compared with batch normalization or no normalization, we find training with weight normalization leads to better accuracy for deep super-resolution networks. Our proposed SR network \textit{WDSR} achieves better results on large-scale DIV2K image super-resolution benchmark in terms of PSNR with same or lower computational complexity. Based on WDSR, our method also won 1st places in NTIRE 2018 Challenge on Single Image Super-Resolution in all three realistic tracks. Experiments and ablation studies support the importance of wide activation for image super-resolution. Code is released at: https://github.com/JiahuiYu/wdsr_ntire2018