PQCAD-DM: Progressive Quantization and Calibration-Assisted Distillation for Extremely Efficient Diffusion Model

TL;DR

This paper introduces PQCAD-DM, a hybrid compression framework that combines progressive quantization and calibration-assisted distillation to significantly improve the efficiency of diffusion models without sacrificing quality.

Contribution

The paper presents a novel hybrid compression method for diffusion models, integrating adaptive two-stage quantization with calibration-guided distillation, enhancing efficiency and performance.

Findings

Halves inference time of diffusion models.

Maintains competitive generative quality with quantized models.

Outperforms fixed-bit quantization methods across datasets.

Abstract

Diffusion models excel in image generation but are computational and resource-intensive due to their reliance on iterative Markov chain processes, leading to error accumulation and limiting the effectiveness of naive compression techniques. In this paper, we propose PQCAD-DM, a novel hybrid compression framework combining Progressive Quantization (PQ) and Calibration-Assisted Distillation (CAD) to address these challenges. PQ employs a two-stage quantization with adaptive bit-width transitions guided by a momentum-based mechanism, reducing excessive weight perturbations in low-precision. CAD leverages full-precision calibration datasets during distillation, enabling the student to match full-precision performance even with a quantized teacher. As a result, PQCAD-DM achieves a balance between computational efficiency and generative quality, halving inference time while maintaining competitive performance. Extensive experiments validate PQCAD-DM's superior generative capabilities and efficiency across diverse datasets, outperforming fixed-bit quantization methods.