FedMR: Fedreated Learning via Model Recombination

TL;DR

FedMR introduces a novel federated learning approach that recombines local models layer-wise to enhance inference accuracy and address limitations of traditional averaging methods, without increasing communication costs.

Contribution

The paper proposes FedMR, a new federated learning paradigm that uses layer-wise model recombination to improve inference performance over existing methods.

Findings

FedMR significantly outperforms state-of-the-art FL methods in inference accuracy.

FedMR achieves these improvements without additional communication overhead.

FedMR effectively finds a globally optimal model through fine-grained recombination.

Abstract

As a promising privacy-preserving machine learning method, Federated Learning (FL) enables global model training across clients without compromising their confidential local data. However, existing FL methods suffer from the problem of low inference performance for unevenly distributed data, since most of them rely on Federated Averaging (FedAvg)-based aggregation. By averaging model parameters in a coarse manner, FedAvg eclipses the individual characteristics of local models, which strongly limits the inference capability of FL. Worse still, in each round of FL training, FedAvg dispatches the same initial local models to clients, which can easily result in stuck-at-local-search for optimal global models. To address the above issues, this paper proposes a novel and effective FL paradigm named FedMR (Federating Model Recombination). Unlike conventional FedAvg-based methods, the cloud server of FedMR shuffles each layer of collected local models and recombines them to achieve new models for local training on clients. Due to the fine-grained model recombination and local training in each FL round, FedMR can quickly figure out one globally optimal model for all the clients. Comprehensive experimental results demonstrate that, compared with state-of-the-art FL methods, FedMR can significantly improve the inference accuracy without causing extra communication overhead.