Enhancing Generalization in Data-free Quantization via Mixup-class Prompting

TL;DR

This paper introduces a mixup-class prompt strategy for data-free quantization that enhances model generalization and stability, leading to improved accuracy especially in low-bit scenarios.

Contribution

It proposes a novel mixup-based text prompting method for synthetic data generation, improving data diversity and quantization performance over existing methods.

Findings

Outperforms state-of-the-art DFQ methods like GenQ.

Achieves new SOTA accuracy in 2-bit weight, 4-bit activation quantization.

Enhances generalization and optimization stability in PTQ.

Abstract

Post-training quantization (PTQ) improves efficiency but struggles with limited calibration data, especially under privacy constraints. Data-free quantization (DFQ) mitigates this by generating synthetic images using generative models such as generative adversarial networks (GANs) and text-conditioned latent diffusion models (LDMs), while applying existing PTQ algorithms. However, the relationship between generated synthetic images and the generalizability of the quantized model during PTQ remains underexplored. Without investigating this relationship, synthetic images generated by previous prompt engineering methods based on single-class prompts suffer from issues such as polysemy, leading to performance degradation. We propose \textbf{mixup-class prompt}, a mixup-based text prompting strategy that fuses multiple class labels at the text prompt level to generate diverse, robust synthetic data. This approach enhances generalization, and improves optimization stability in PTQ. We provide quantitative insights through gradient norm and generalization error analysis. Experiments on convolutional neural networks (CNNs) and vision transformers (ViTs) show that our method consistently outperforms state-of-the-art DFQ methods like GenQ. Furthermore, it pushes the performance boundary in extremely low-bit scenarios, achieving new state-of-the-art accuracy in challenging 2-bit weight, 4-bit activation (W2A4) quantization.