AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution

TL;DR

AdaBM introduces a rapid, on-the-fly adaptive quantization method for image super-resolution that significantly reduces processing time while maintaining competitive accuracy, enabling more practical applications.

Contribution

It presents the first fast, on-the-fly adaptive quantization framework for SR that requires minimal calibration and no extensive retraining.

Findings

Processing time reduced by 2000x

Achieves competitive accuracy with previous methods

Requires only a small set of calibration images

Abstract

Although image super-resolution (SR) problem has experienced unprecedented restoration accuracy with deep neural networks, it has yet limited versatile applications due to the substantial computational costs. Since different input images for SR face different restoration difficulties, adapting computational costs based on the input image, referred to as adaptive inference, has emerged as a promising solution to compress SR networks. Specifically, adapting the quantization bit-widths has successfully reduced the inference and memory cost without sacrificing the accuracy. However, despite the benefits of the resultant adaptive network, existing works rely on time-intensive quantization-aware training with full access to the original training pairs to learn the appropriate bit allocation policies, which limits its ubiquitous usage. To this end, we introduce the first on-the-fly adaptive quantization framework that accelerates the processing time from hours to seconds. We formulate the bit allocation problem with only two bit mapping modules: one to map the input image to the image-wise bit adaptation factor and one to obtain the layer-wise adaptation factors. These bit mappings are calibrated and fine-tuned using only a small number of calibration images. We achieve competitive performance with the previous adaptive quantization methods, while the processing time is accelerated by x2000. Codes are available at https://github.com/Cheeun/AdaBM.