Frequency Composition for Compressed and Domain-Adaptive Neural Networks

TL;DR

This paper introduces CoDA, a frequency composition framework that unifies neural network compression and domain adaptation, enabling resource-efficient models to adapt to domain shifts during testing.

Contribution

CoDA integrates frequency-based training and test-time adaptation, allowing compressed models to handle domain shifts without requiring source data during testing.

Findings

Achieves 7.96% accuracy improvement on CIFAR10-C

Achieves 5.37% accuracy improvement on ImageNet-C

Effective integration with existing QAT and TTA methods

Abstract

Modern on-device neural network applications must operate under resource constraints while adapting to unpredictable domain shifts. However, this combined challenge-model compression and domain adaptation-remains largely unaddressed, as prior work has tackled each issue in isolation: compressed networks prioritize efficiency within a fixed domain, whereas large, capable models focus on handling domain shifts. In this work, we propose CoDA, a frequency composition-based framework that unifies compression and domain adaptation. During training, CoDA employs quantization-aware training (QAT) with low-frequency components, enabling a compressed model to selectively learn robust, generalizable features. At test time, it refines the compact model in a source-free manner (i.e., test-time adaptation, TTA), leveraging the full-frequency information from incoming data to adapt to target domains while treating high-frequency components as domain-specific cues. LFC are aligned with the trained distribution, while HFC unique to the target distribution are solely utilized for batch normalization. CoDA can be integrated synergistically into existing QAT and TTA methods. CoDA is evaluated on widely used domain-shift benchmarks, including CIFAR10-C and ImageNet-C, across various model architectures. With significant compression, it achieves accuracy improvements of 7.96%p on CIFAR10-C and 5.37%p on ImageNet-C over the full-precision TTA baseline.