Provably Personalized and Robust Federated Learning

TL;DR

This paper introduces provably optimal algorithms for personalized federated learning that cluster clients based on objectives, achieving robust convergence even with malicious clients, and formalizes the problem with strong theoretical guarantees.

Contribution

It formalizes the clustering-based personalization problem in federated learning with provable convergence guarantees and robustness, proposing simple iterative algorithms for client clustering and model training.

Findings

Achieves optimal convergence rates matching true clustering knowledge.

Provides algorithms that are robust to Byzantine (malicious) clients.

Formalizes the problem with strong theoretical guarantees.

Abstract

Identifying clients with similar objectives and learning a model-per-cluster is an intuitive and interpretable approach to personalization in federated learning. However, doing so with provable and optimal guarantees has remained an open challenge. We formalize this problem as a stochastic optimization problem, achieving optimal convergence rates for a large class of loss functions. We propose simple iterative algorithms which identify clusters of similar clients and train a personalized model-per-cluster, using local client gradients and flexible constraints on the clusters. The convergence rates of our algorithms asymptotically match those obtained if we knew the true underlying clustering of the clients and are provably robust in the Byzantine setting where some fraction of the clients are malicious.