CR-QAT: Curriculum Relational Quantization-Aware Training for Open-Vocabulary Object Detection

TL;DR

This paper introduces CR-QAT, a novel training framework that combines curriculum-based quantization and relational knowledge distillation to effectively compress open-vocabulary object detection models without sacrificing fine-grained vision-language alignment.

Contribution

CR-QAT is the first integrated approach that addresses quantization errors in open-vocabulary detection by progressive model partitioning and relational knowledge transfer.

Findings

CR-QAT outperforms existing QAT methods under low-bit settings.

Achieves up to 38.9% and 40.9% relative AP improvements on LVIS and COCO.

Effectively preserves vision-language alignment during aggressive quantization.

Abstract

Open-vocabulary object detection (OVOD) enables novel category detection via vision-language alignment, but massive model sizes hinder deployment on resource-constrained devices. While quantization offers practical compression, we reveal that naive extreme low-bit (e.g., 4-bit) quantization severely degrades fine-grained vision-language alignment and distorts inter-region relational structures. To address this, we propose curriculum relational quantization-aware training (CR-QAT), an integrated framework combining stage-by-stage optimization with relational knowledge distillation. Within CR-QAT, curriculum QAT (CQAT) mitigates error accumulation by partitioning the model for progressive quantization, ensuring stable optimization via error isolation. Concurrently, text-centric relational KD (TRKD) is applied to task-relevant modules. By constructing text-anchored pairwise similarity matrices, TRKD comprehensively transfers the teacher's multi-dimensional relational knowledge. Experiments on LVIS and COCO zero-shot benchmarks demonstrate that CR-QAT consistently outperforms existing QAT baselines under aggressive low-bit settings, achieving relative AP improvements of up to 38.9% and 40.9%, respectively.