Contrastive Test-Time Adaptation

TL;DR

AdaContrast introduces a contrastive test-time adaptation method that leverages self-supervised learning and pseudo label refinement to improve model adaptation to new domains without source data.

Contribution

The paper presents AdaContrast, a novel approach combining contrastive learning with pseudo label refinement for effective, memory-efficient test-time adaptation.

Findings

Achieves state-of-the-art results on major benchmarks.

Demonstrates robustness to hyper-parameter variations.

Improves model calibration and memory efficiency.

Abstract

Test-time adaptation is a special setting of unsupervised domain adaptation where a trained model on the source domain has to adapt to the target domain without accessing source data. We propose a novel way to leverage self-supervised contrastive learning to facilitate target feature learning, along with an online pseudo labeling scheme with refinement that significantly denoises pseudo labels. The contrastive learning task is applied jointly with pseudo labeling, contrasting positive and negative pairs constructed similarly as MoCo but with source-initialized encoder, and excluding same-class negative pairs indicated by pseudo labels. Meanwhile, we produce pseudo labels online and refine them via soft voting among their nearest neighbors in the target feature space, enabled by maintaining a memory queue. Our method, AdaContrast, achieves state-of-the-art performance on major benchmarks while having several desirable properties compared to existing works, including memory efficiency, insensitivity to hyper-parameters, and better model calibration. Project page: sites.google.com/view/adacontrast.