HarmoQ: Harmonized Post-Training Quantization for High-Fidelity Image

TL;DR

HarmoQ introduces a unified post-training quantization framework that harmonizes weight and activation quantization, significantly improving super-resolution model performance under aggressive compression while reducing computational resources.

Contribution

This work provides the first systematic analysis of weight-activation coupling in super-resolution quantization and proposes a novel harmonized approach for high-fidelity image restoration.

Findings

Outperforms prior methods by 0.46 dB on Set5 at 2-bit quantization

Achieves 3.2x speedup and 4x memory reduction on A100 GPUs

Uncovers the asymmetric impact of weight and activation quantization on image quality

Abstract

Post-training quantization offers an efficient pathway to deploy super-resolution models, yet existing methods treat weight and activation quantization independently, missing their critical interplay. Through controlled experiments on SwinIR, we uncover a striking asymmetry: weight quantization primarily degrades structural similarity, while activation quantization disproportionately affects pixel-level accuracy. This stems from their distinct roles--weights encode learned restoration priors for textures and edges, whereas activations carry input-specific intensity information. Building on this insight, we propose HarmoQ, a unified framework that harmonizes quantization across components through three synergistic steps: structural residual calibration proactively adjusts weights to compensate for activation-induced detail loss, harmonized scale optimization analytically balances quantization difficulty via closed-form solutions, and adaptive boundary refinement iteratively maintains this balance during optimization. Experiments show HarmoQ achieves substantial gains under aggressive compression, outperforming prior art by 0.46 dB on Set5 at 2-bit while delivering 3.2x speedup and 4x memory reduction on A100 GPUs. This work provides the first systematic analysis of weight-activation coupling in super-resolution quantization and establishes a principled solution for efficient high-quality image restoration.