Outlier-Aware Post-Training Quantization for Image Super-Resolution

TL;DR

This paper introduces a novel outlier-aware post-training quantization method for image super-resolution that partitions activations into regions and applies sensitivity-aware finetuning, significantly improving performance and speed.

Contribution

It proposes a dual-region quantization strategy combined with sensitivity-aware finetuning to enhance PTQ performance for SR networks, addressing outliers and layer sensitivities.

Findings

Outlier-aware quantization improves SR performance.

Method achieves comparable results to QAT with 75x speedup.

Outperforms existing PTQ methods across datasets.

Abstract

Quantization techniques, including quantization-aware training (QAT) and post-training quantization (PTQ), have become essential for inference acceleration of image super-resolution (SR) networks. Compared to QAT, PTQ has garnered significant attention as it eliminates the need for ground truth and model retraining. However, existing PTQ methods for SR often fail to achieve satisfactory performance as they overlook the impact of outliers in activation. Our empirical analysis reveals that these prevalent activation outliers are strongly correlated with image color information, and directly removing them leads to significant performance degradation. Motivated by this, we propose a dual-region quantization strategy that partitions activations into an outlier region and a dense region, applying uniform quantization to each region independently to better balance bit-width allocation. Furthermore, we observe that different network layers exhibit varying sensitivities to quantization, leading to different levels of performance degradation. To address this, we introduce sensitivity-aware finetuning that encourages the model to focus more on highly sensitive layers, further enhancing quantization performance. Extensive experiments demonstrate that our method outperforms existing PTQ approaches across various SR networks and datasets, while achieving performance comparable to QAT methods in most scenarios with at least a 75 speedup.