FedFMC: Sequential Efficient Federated Learning on Non-iid Data

TL;DR

FedFMC introduces a dynamic fork-merge-consolidate approach for federated learning, significantly improving performance on non-iid data without extra communication or shared data, addressing key limitations of existing methods.

Contribution

The paper presents FedFMC, a novel federated learning method that adaptively forks and merges models to handle non-iid data efficiently without additional communication overhead.

Findings

FedFMC outperforms baseline methods on non-iid datasets.

It does not require a globally shared data subset.

It maintains communication costs comparable to standard approaches.

Abstract

As a mechanism for devices to update a global model without sharing data, federated learning bridges the tension between the need for data and respect for privacy. However, classic FL methods like Federated Averaging struggle with non-iid data, a prevalent situation in the real world. Previous solutions are sub-optimal as they either employ a small shared global subset of data or greater number of models with increased communication costs. We propose FedFMC (Fork-Merge-Consolidate), a method that dynamically forks devices into updating different global models then merges and consolidates separate models into one. We first show the soundness of FedFMC on simple datasets, then run several experiments comparing against baseline approaches. These experiments show that FedFMC substantially improves upon earlier approaches to non-iid data in the federated learning context without using a globally shared subset of data nor increase communication costs.