Federated Meta-Learning with Fast Convergence and Efficient Communication

TL;DR

This paper introduces FedMeta, a federated meta-learning framework that significantly reduces communication costs, accelerates convergence, and improves accuracy in distributed mobile device training while preserving user privacy.

Contribution

The paper proposes FedMeta, a novel federated meta-learning approach that replaces global models with a shared parameterized algorithm, enhancing efficiency and privacy.

Findings

FedMeta reduces communication costs by 2.82-4.33 times.

FedMeta achieves 3.23%-14.84% higher accuracy than FedAvg.

FedMeta converges faster in empirical evaluations.

Abstract

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.