Federated Learning with Flexible Control

TL;DR

This paper introduces FlexFL, a flexible federated learning algorithm that jointly optimizes local computation and communication to improve efficiency, supported by theoretical convergence guarantees and experimental validation.

Contribution

It proposes a novel FL algorithm with adjustable control parameters for computation and communication, along with a stochastic optimization method to tune these parameters for optimal performance.

Findings

FlexFL achieves efficient resource utilization in federated learning.

Theoretical convergence bounds are established for the proposed algorithm.

Experimental results validate the effectiveness of the control optimization.

Abstract

Federated learning (FL) enables distributed model training from local data collected by users. In distributed systems with constrained resources and potentially high dynamics, e.g., mobile edge networks, the efficiency of FL is an important problem. Existing works have separately considered different configurations to make FL more efficient, such as infrequent transmission of model updates, client subsampling, and compression of update vectors. However, an important open problem is how to jointly apply and tune these control knobs in a single FL algorithm, to achieve the best performance by allowing a high degree of freedom in control decisions. In this paper, we address this problem and propose FlexFL - an FL algorithm with multiple options that can be adjusted flexibly. Our FlexFL algorithm allows both arbitrary rates of local computation at clients and arbitrary amounts of communication between clients and the server, making both the computation and communication resource consumption adjustable. We prove a convergence upper bound of this algorithm. Based on this result, we further propose a stochastic optimization formulation and algorithm to determine the control decisions that (approximately) minimize the convergence bound, while conforming to constraints related to resource consumption. The advantage of our approach is also verified using experiments.