RTF-Q: Efficient Unsupervised Domain Adaptation with Retraining-free Quantization

TL;DR

RTF-Q introduces a retraining-free quantization method for unsupervised domain adaptation on resource-limited devices, enabling efficient, multi-bitwidth adaptation with minimal training overhead and competitive accuracy.

Contribution

The paper presents a novel retraining-free quantization approach that adapts pre-trained models to multiple device constraints using weight-sharing and multi-bitwidth training.

Findings

Achieves competitive accuracy with state-of-the-art methods

Reduces memory and computational costs significantly

Supports dynamic computation budgets on edge devices

Abstract

Performing unsupervised domain adaptation on resource-constrained edge devices is challenging. Existing research typically adopts architecture optimization (e.g., designing slimmable networks) but requires expensive training costs. Moreover, it does not consider the considerable precision redundancy of parameters and activations. To address these limitations, we propose efficient unsupervised domain adaptation with ReTraining-Free Quantization (RTF-Q). Our approach uses low-precision quantization architectures with varying computational costs, adapting to devices with dynamic computation budgets. We subtly configure subnet dimensions and leverage weight-sharing to optimize multiple architectures within a single set of weights, enabling the use of pre-trained models from open-source repositories. Additionally, we introduce multi-bitwidth joint training and the SandwichQ rule, both of which are effective in handling multiple quantization bit-widths across subnets. Experimental results demonstrate that our network achieves competitive accuracy with state-of-the-art methods across three benchmarks while significantly reducing memory and computational costs.