Resource-Efficient and Delay-Aware Federated Learning Design under Edge Heterogeneity

TL;DR

This paper introduces StoFedDelAv, a federated learning algorithm designed to optimize training efficiency and delay-awareness under device heterogeneity and network delays, with proven convergence and improved performance.

Contribution

It proposes a novel FL algorithm with a local-global model combiner, analyzes its convergence, and optimizes device minibatch sizes considering energy and delay constraints.

Findings

StoFedDelAv outperforms existing FL methods in simulations.

Optimal combiner weights account for delay and gradient error.

Network-aware minibatch tuning reduces energy and training loss.

Abstract

Federated learning (FL) has emerged as a popular technique for distributing machine learning across wireless edge devices. We examine FL under two salient properties of contemporary networks: device-server communication delays and device computation heterogeneity. Our proposed StoFedDelAv algorithm incorporates a local-global model combiner into the FL synchronization step. We theoretically characterize the convergence behavior of StoFedDelAv and obtain the optimal combiner weights, which consider the global model delay and expected local gradient error at each device. We then formulate a network-aware optimization problem which tunes the minibatch sizes of the devices to jointly minimize energy consumption and machine learning training loss, and solve the non-convex problem through a series of convex approximations. Our simulations reveal that StoFedDelAv outperforms the current art in FL, evidenced by the obtained improvements in optimization objective.