PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution

TL;DR

PassionSR introduces a post-training quantization method with adaptive scaling for one-step diffusion-based image super-resolution, achieving comparable quality to full-precision models with reduced computational and storage costs.

Contribution

It proposes a novel quantization approach with adaptive scale, including LBQ, LET, and DQC strategies, to efficiently quantize diffusion-based SR models in one step.

Findings

8-bit and 6-bit PassionSR achieve comparable results to full-precision models.

PassionSR outperforms recent low-bit quantization methods for image SR.

The method enables efficient deployment on hardware devices.

Abstract

Diffusion-based image super-resolution (SR) models have shown superior performance at the cost of multiple denoising steps. However, even though the denoising step has been reduced to one, they require high computational costs and storage requirements, making it difficult for deployment on hardware devices. To address these issues, we propose a novel post-training quantization approach with adaptive scale in one-step diffusion (OSD) image SR, PassionSR. First, we simplify OSD model to two core components, UNet and Variational Autoencoder (VAE) by removing the CLIPEncoder. Secondly, we propose Learnable Boundary Quantizer (LBQ) and Learnable Equivalent Transformation (LET) to optimize the quantization process and manipulate activation distributions for better quantization. Finally, we design a Distributed Quantization Calibration (DQC) strategy that stabilizes the training of quantized parameters for rapid convergence. Comprehensive experiments demonstrate that PassionSR with 8-bit and 6-bit obtains comparable visual results with full-precision model. Moreover, our PassionSR achieves significant advantages over recent leading low-bit quantization methods for image SR. Our code will be at https://github.com/libozhu03/PassionSR.