Multi-Model Federated Learning with Provable Guarantees

TL;DR

This paper introduces two new variants of the FedAvg algorithm for multi-model federated learning, providing convergence guarantees and demonstrating improved performance over separate training through theoretical analysis and experiments.

Contribution

It proposes novel FedAvg variants for multi-model FL with provable convergence, enhancing efficiency and performance over traditional methods.

Findings

Multi-model FL can outperform separate training for the same computation.

The proposed algorithms have provable convergence guarantees.

Experimental results validate theoretical claims across various convexity settings.

Abstract

Federated Learning (FL) is a variant of distributed learning where edge devices collaborate to learn a model without sharing their data with the central server or each other. We refer to the process of training multiple independent models simultaneously in a federated setting using a common pool of clients as multi-model FL. In this work, we propose two variants of the popular FedAvg algorithm for multi-model FL, with provable convergence guarantees. We further show that for the same amount of computation, multi-model FL can have better performance than training each model separately. We supplement our theoretical results with experiments in strongly convex, convex, and non-convex settings.