Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

TL;DR

This paper analyzes the impact of data heterogeneity on wireless federated learning and proposes an optimization framework for client scheduling and resource allocation to improve learning accuracy and energy efficiency.

Contribution

It introduces a closed-form upper bound on FL loss considering data heterogeneity and develops an optimization method for resource allocation and scheduling under energy and latency constraints.

Findings

Proposed algorithm outperforms benchmarks in accuracy.

Efficient resource allocation reduces energy consumption.

Optimization improves training performance under heterogeneity.

Abstract

With the rapid proliferation of smart mobile devices, federated learning (FL) has been widely considered for application in wireless networks for distributed model training. However, data heterogeneity, e.g., non-independently identically distributions and different sizes of training data among clients, poses major challenges to wireless FL. Limited communication resources complicate the implementation of fair scheduling which is required for training on heterogeneous data, and further deteriorate the overall performance. To address this issue, this paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. Specifically, we first develop a closed-form expression for an upper bound on the FL loss function, with a particular emphasis on data heterogeneity described by a dataset size vector and a data divergence vector. Then we formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE). Next, via the Lyapunov drift technique, we transform the CRE optimization problem into a series of tractable problems. Extensive experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.