UPFL: Unsupervised Personalized Federated Learning towards New Clients

TL;DR

This paper introduces FedTTA, an unsupervised personalized federated learning method that effectively adapts models for new clients with unlabeled data, addressing data heterogeneity and device differences.

Contribution

We extend adaptive risk minimization to unsupervised federated learning and propose FedTTA with optimization strategies and knowledge distillation for new client adaptation.

Findings

FedTTA outperforms eleven baselines on five datasets.

The proposed methods effectively handle unlabeled new clients.

Knowledge distillation improves device heterogeneity adaptation.

Abstract

Personalized federated learning has gained significant attention as a promising approach to address the challenge of data heterogeneity. In this paper, we address a relatively unexplored problem in federated learning. When a federated model has been trained and deployed, and an unlabeled new client joins, providing a personalized model for the new client becomes a highly challenging task. To address this challenge, we extend the adaptive risk minimization technique into the unsupervised personalized federated learning setting and propose our method, FedTTA. We further improve FedTTA with two simple yet effective optimization strategies: enhancing the training of the adaptation model with proxy regularization and early-stopping the adaptation through entropy. Moreover, we propose a knowledge distillation loss specifically designed for FedTTA to address the device heterogeneity. Extensive experiments on five datasets against eleven baselines demonstrate the effectiveness of our proposed FedTTA and its variants. The code is available at: https://github.com/anonymous-federated-learning/code.