Tackling Computational Heterogeneity in FL: A Few Theoretical Insights

TL;DR

This paper introduces a new aggregation framework for federated learning that addresses computational heterogeneity among clients, supported by theoretical analysis and experimental validation.

Contribution

It proposes a novel aggregation method specifically designed to handle heterogeneity in data and computation in federated learning, with thorough theoretical and experimental analysis.

Findings

The proposed algorithms effectively manage heterogeneity in federated settings.

Theoretical analysis confirms convergence properties under heterogeneity.

Experimental results demonstrate improved model performance with the new framework.

Abstract

The future of machine learning lies in moving data collection along with training to the edge. Federated Learning, for short FL, has been recently proposed to achieve this goal. The principle of this approach is to aggregate models learned over a large number of distributed clients, i.e., resource-constrained mobile devices that collect data from their environment, to obtain a new more general model. The latter is subsequently redistributed to clients for further training. A key feature that distinguishes federated learning from data-center-based distributed training is the inherent heterogeneity. In this work, we introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneity in federated optimization, in terms of both heterogeneous data and local updates. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.