Flow: Per-Instance Personalized Federated Learning Through Dynamic Routing

TL;DR

Flow introduces a fine-grained, per-instance personalized federated learning method that dynamically routes inputs to either local or global models, improving accuracy without maintaining client state across rounds.

Contribution

This work presents Flow, a novel stateless, per-instance routing approach for personalized federated learning, enhancing accuracy and scalability over existing methods.

Findings

Flow outperforms state-of-the-art personalized FL methods on multiple datasets.

Flow achieves higher prediction accuracy with dynamic per-instance routing.

Flow is practical for large-scale FL and supports new clients efficiently.

Abstract

Personalization in Federated Learning (FL) aims to modify a collaboratively trained global model according to each client. Current approaches to personalization in FL are at a coarse granularity, i.e. all the input instances of a client use the same personalized model. This ignores the fact that some instances are more accurately handled by the global model due to better generalizability. To address this challenge, this work proposes Flow, a fine-grained stateless personalized FL approach. Flow creates dynamic personalized models by learning a routing mechanism that determines whether an input instance prefers the local parameters or its global counterpart. Thus, Flow introduces per-instance routing in addition to leveraging per-client personalization to improve accuracies at each client. Further, Flow is stateless which makes it unnecessary for a client to retain its personalized state across FL rounds. This makes Flow practical for large-scale FL settings and friendly to newly joined clients. Evaluations on Stackoverflow, Reddit, and EMNIST datasets demonstrate the superiority in prediction accuracy of Flow over state-of-the-art non-personalized and only per-client personalized approaches to FL.