Fully Quantized Image Super-Resolution Networks

TL;DR

This paper introduces a fully quantized super-resolution network that maintains high accuracy while significantly reducing computational cost and memory usage, addressing key challenges in low-bit quantization for SR models.

Contribution

It proposes a novel end-to-end quantization framework for all layers, including skip connections, and introduces techniques to overcome training difficulties in low-bit SR networks.

Findings

Achieves comparable performance to full-precision models on benchmark datasets.

Surpasses existing quantized SR methods in accuracy and efficiency.

Reduces computational cost and memory consumption significantly.

Abstract

With the rising popularity of intelligent mobile devices, it is of great practical significance to develop accurate, realtime and energy-efficient image Super-Resolution (SR) inference methods. A prevailing method for improving the inference efficiency is model quantization, which allows for replacing the expensive floating-point operations with efficient fixed-point or bitwise arithmetic. To date, it is still challenging for quantized SR frameworks to deliver feasible accuracy-efficiency trade-off. Here, we propose a Fully Quantized image Super-Resolution framework (FQSR) to jointly optimize efficiency and accuracy. In particular, we target on obtaining end-to-end quantized models for all layers, especially including skip connections, which was rarely addressed in the literature. We further identify training obstacles faced by low-bit SR networks and propose two novel methods accordingly. The two difficulites are caused by 1) activation and weight distributions being vastly distinctive in different layers; 2) the inaccurate approximation of the quantization. We apply our quantization scheme on multiple mainstream super-resolution architectures, including SRResNet, SRGAN and EDSR. Experimental results show that our FQSR using low bits quantization can achieve on par performance compared with the full-precision counterparts on five benchmark datasets and surpass state-of-the-art quantized SR methods with significantly reduced computational cost and memory consumption.