FedMe: Federated Learning via Model Exchange

TL;DR

FedMe introduces a novel federated learning approach where clients exchange models to automatically tune architectures and improve personalization, effectively handling data heterogeneity without central data access.

Contribution

The paper proposes FedMe, a federated learning method enabling automatic model architecture tuning through model exchange and deep mutual learning, addressing data heterogeneity challenges.

Findings

FedMe outperforms existing federated learning methods on three real datasets.

Clients effectively tune their models automatically during training.

Model exchange facilitates personalized architecture optimization.

Abstract

Federated learning is a distributed machine learning method in which a single server and multiple clients collaboratively build machine learning models without sharing datasets on clients. Numerous methods have been proposed to cope with the data heterogeneity issue in federated learning. Existing solutions require a model architecture tuned by the central server, yet a major technical challenge is that it is difficult to tune the model architecture due to the absence of local data on the central server. In this paper, we propose Federated learning via Model exchange (FedMe), which personalizes models with automatic model architecture tuning during the learning process. The novelty of FedMe lies in its learning process: clients exchange their models for model architecture tuning and model training. First, to optimize the model architectures for local data, clients tune their own personalized models by comparing to exchanged models and picking the one that yields the best performance. Second, clients train both personalized models and exchanged models by using deep mutual learning, in spite of different model architectures across the clients. We perform experiments on three real datasets and show that FedMe outperforms state-of-the-art federated learning methods while tuning model architectures automatically.