On-Demand Unlabeled Personalized Federated Learning

TL;DR

This paper introduces On-Demand Unlabeled Personalized Federated Learning (OD-PFL), enabling models to adapt to new unlabeled clients post-deployment using a hypernetwork approach, improving generalization especially with large client shifts.

Contribution

The paper proposes a novel framework, ODPFL-HN, for adapting federated models to unlabeled new clients using a hypernetwork and client representations, addressing a practical deployment scenario.

Findings

ODPFL-HN outperforms existing FL and PFL methods on benchmark datasets.

The approach effectively handles large distribution shifts of new clients.

The method allows for differential privacy in client adaptation.

Abstract

In Federated Learning (FL), multiple clients collaborate to learn a shared model through a central server while keeping data decentralized. Personalized Federated Learning (PFL) further extends FL by learning a personalized model per client. In both FL and PFL, all clients participate in the training process and their labeled data are used for training. However, in reality, novel clients may wish to join a prediction service after it has been deployed, obtaining predictions for their own \textbf{unlabeled} data. Here, we introduce a new learning setup, On-Demand Unlabeled PFL (OD-PFL), where a system trained on a set of clients, needs to be later applied to novel unlabeled clients at inference time. We propose a novel approach to this problem, ODPFL-HN, which learns to produce a new model for the late-to-the-party client. Specifically, we train an encoder network that learns a representation for a client given its unlabeled data. That client representation is fed to a hypernetwork that generates a personalized model for that client. Evaluated on five benchmark datasets, we find that ODPFL-HN generalizes better than the current FL and PFL methods, especially when the novel client has a large shift from training clients. We also analyzed the generalization error for novel clients, and showed analytically and experimentally how novel clients can apply differential privacy.