TTAQ: Towards Stable Post-training Quantization in Continuous Domain Adaptation

TL;DR

TTAQ introduces a stable post-training quantization method for test-time adaptation that effectively handles domain shifts and class imbalance, significantly improving low-bit model accuracy in dynamic real-world scenarios.

Contribution

The paper proposes TTAQ, a novel stable quantization framework with PEM, PCR, and ABL to enhance PTQ performance under continual domain shifts.

Findings

TTAQ reduces mean error of 2-bit models on ImageNet-C by 10.1%.

TTAQ outperforms existing baselines in dynamic test domains.

The method effectively mitigates domain shift and class imbalance issues.

Abstract

Post-training quantization (PTQ) reduces excessive hardware cost by quantizing full-precision models into lower bit representations on a tiny calibration set, without retraining. Despite the remarkable progress made through recent efforts, traditional PTQ methods typically encounter failure in dynamic and ever-changing real-world scenarios, involving unpredictable data streams and continual domain shifts, which poses greater challenges. In this paper, we propose a novel and stable quantization process for test-time adaptation (TTA), dubbed TTAQ, to address the performance degradation of traditional PTQ in dynamically evolving test domains. To tackle domain shifts in quantizer, TTAQ proposes the Perturbation Error Mitigation (PEM) and Perturbation Consistency Reconstruction (PCR). Specifically, PEM analyzes the error propagation and devises a weight regularization scheme to mitigate the impact of input perturbations. On the other hand, PCR introduces consistency learning to ensure that quantized models provide stable predictions for same sample. Furthermore, we introduce Adaptive Balanced Loss (ABL) to adjust the logits by taking advantage of the frequency and complexity of the class, which can effectively address the class imbalance caused by unpredictable data streams during optimization. Extensive experiments are conducted on multiple datasets with generic TTA methods, proving that TTAQ can outperform existing baselines and encouragingly improve the accuracy of low bit PTQ models in continually changing test domains. For instance, TTAQ decreases the mean error of 2-bit models on ImageNet-C dataset by an impressive 10.1\%.