Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning

TL;DR

This paper introduces Divide and Contrast (DaC), a novel source-free domain adaptation method that combines global class clustering and local structure learning through adaptive contrastive learning, improving performance across multiple datasets.

Contribution

DaC uniquely divides target data based on source model confidence and applies tailored contrastive learning strategies, effectively addressing limitations of existing SFUDA methods.

Findings

DaC outperforms state-of-the-art methods on VisDA, Office-Home, and DomainNet datasets.

The adaptive division improves domain alignment and class clustering.

Memory bank-based MMD reduces distribution mismatch effectively.

Abstract

We investigate a practical domain adaptation task, called source-free domain adaptation (SFUDA), where the source-pretrained model is adapted to the target domain without access to the source data. Existing techniques mainly leverage self-supervised pseudo labeling to achieve class-wise global alignment [1] or rely on local structure extraction that encourages feature consistency among neighborhoods [2]. While impressive progress has been made, both lines of methods have their own drawbacks - the "global" approach is sensitive to noisy labels while the "local" counterpart suffers from source bias. In this paper, we present Divide and Contrast (DaC), a new paradigm for SFUDA that strives to connect the good ends of both worlds while bypassing their limitations. Based on the prediction confidence of the source model, DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals under an adaptive contrastive learning framework. Specifically, the source-like samples are utilized for learning global class clustering thanks to their relatively clean labels. The more noisy target-specific data are harnessed at the instance level for learning the intrinsic local structures. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch. Extensive experiments on VisDA, Office-Home, and the more challenging DomainNet have verified the superior performance of DaC over current state-of-the-art approaches. The code is available at https://github.com/ZyeZhang/DaC.git.