A Prototype-Oriented Clustering for Domain Shift with Source Privacy

TL;DR

This paper proposes Prototype-oriented Clustering with Distillation (PCD), a method that enhances unsupervised clustering under domain shift while protecting source data and model privacy, demonstrating strong experimental results.

Contribution

Introduces PCD, a novel source-private clustering approach that aligns prototypes, distills knowledge, and refines models without exposing source data or models.

Findings

Effective across multiple benchmarks

Improves clustering performance under domain shift

Ensures source data privacy

Abstract

Unsupervised clustering under domain shift (UCDS) studies how to transfer the knowledge from abundant unlabeled data from multiple source domains to learn the representation of the unlabeled data in a target domain. In this paper, we introduce Prototype-oriented Clustering with Distillation (PCD) to not only improve the performance and applicability of existing methods for UCDS, but also address the concerns on protecting the privacy of both the data and model of the source domains. PCD first constructs a source clustering model by aligning the distributions of prototypes and data. It then distills the knowledge to the target model through cluster labels provided by the source model while simultaneously clustering the target data. Finally, it refines the target model on the target domain data without guidance from the source model. Experiments across multiple benchmarks show the effectiveness and generalizability of our source-private clustering method.