COSMOS: Model-Agnostic Personalized Federated Learning with Clustered Server Models and Pseudo-Label-Only Communication

TL;DR

COSMOS introduces a model-agnostic federated learning framework that uses pseudo-label communication and client clustering to achieve personalized, scalable models in heterogeneous environments.

Contribution

It provides the first theoretical analysis of distillation for personalization and demonstrates superior empirical performance over existing baselines.

Findings

COSMOS outperforms all model-agnostic FL baselines in benchmarks.

Theoretical analysis shows exponential contraction of personalization risk.

COSMOS remains competitive with state-of-the-art personalized FL methods.

Abstract

Federated learning (FL) in heterogeneous environments remains challenging because client models often differ in both architecture and data distribution. While recent approaches attempt to address this challenge through client clustering and knowledge distillation, simultaneously handling architectural and statistical heterogeneity remains difficult. We introduce COSMOS, a model-agnostic framework that enables server-side personalization using only pseudo-label communication. Clients train local models and predict on the public data; the server clusters clients by prediction similarity, trains a cluster-specific model for each group using its own compute, and distills the resulting models back to clients. We provide the first theoretical analysis showing that distillation from the learned cluster models can yield exponential personalization risk contraction, going beyond the convergence-to-stationarity guarantees typically provided in model-agnostic FL. Experiments across benchmarks demonstrate that COSMOS consistently outperforms all model-agnostic FL baselines while remaining competitive with state-of-the-art personalized FL methods. More broadly, our results highlight personalized server-side learning with pseudo-labels as a promising paradigm for scalable and model-agnostic federated learning in highly heterogeneous environments.