Multi-Center Federated Learning: Clients Clustering for Better Personalization

TL;DR

This paper introduces a multi-center federated learning approach that learns multiple global models and matches users to centers, improving personalization and handling data heterogeneity better than single-model methods.

Contribution

It proposes a novel multi-center aggregation mechanism with an EM algorithm to better capture data heterogeneity in federated learning.

Findings

Outperforms existing federated learning methods on benchmark datasets.

Effectively captures user data heterogeneity through multiple centers.

Improves personalization in federated learning applications.

Abstract

Federated learning has received great attention for its capability to train a large-scale model in a decentralized manner without needing to access user data directly. It helps protect the users' private data from centralized collecting. Unlike distributed machine learning, federated learning aims to tackle non-IID data from heterogeneous sources in various real-world applications, such as those on smartphones. Existing federated learning approaches usually adopt a single global model to capture the shared knowledge of all users by aggregating their gradients, regardless of the discrepancy between their data distributions. However, due to the diverse nature of user behaviors, assigning users' gradients to different global models (i.e., centers) can better capture the heterogeneity of data distributions across users. Our paper proposes a novel multi-center aggregation mechanism for federated learning, which learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. We formulate the problem as a joint optimization that can be efficiently solved by a stochastic expectation maximization (EM) algorithm. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.