Better Methods and Theory for Federated Learning: Compression, Client Selection and Heterogeneity

TL;DR

This paper advances federated learning by introducing new methods and theoretical insights to address challenges like compression, client selection, and heterogeneity, aiming for practical, privacy-preserving solutions with rigorous guarantees.

Contribution

It proposes novel algorithms and theoretical frameworks to improve federated learning's efficiency, robustness, and privacy preservation in heterogeneous client environments.

Findings

Enhanced algorithms for client selection and model compression.

Theoretical guarantees for convergence and privacy.

Improved performance in heterogeneous federated settings.

Abstract

Federated learning (FL) is an emerging machine learning paradigm involving multiple clients, e.g., mobile phone devices, with an incentive to collaborate in solving a machine learning problem coordinated by a central server. FL was proposed in 2016 by Kone\v{c}n\'{y} et al. and McMahan et al. as a viable privacy-preserving alternative to traditional centralized machine learning since, by construction, the training data points are decentralized and never transferred by the clients to a central server. Therefore, to a certain degree, FL mitigates the privacy risks associated with centralized data collection. Unfortunately, optimization for FL faces several specific issues that centralized optimization usually does not need to handle. In this thesis, we identify several of these challenges and propose new methods and algorithms to address them, with the ultimate goal of enabling practical FL solutions supported with mathematically rigorous guarantees.