Adaptive Deadline and Batch Layered Synchronized Federated Learning

TL;DR

ADEL-FL is a federated learning framework that adaptively optimizes deadlines and batch sizes for each client to improve convergence and accuracy in heterogeneous environments.

Contribution

It introduces a joint optimization approach for deadlines and batch sizes in federated learning, addressing device heterogeneity and strict time constraints.

Findings

Outperforms existing methods in convergence speed

Achieves higher final accuracy under heterogeneity

Provides theoretical convergence guarantees

Abstract

Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner. However, synchronous FL suffers from latency bottlenecks due to device heterogeneity, where slower clients (stragglers) delay or degrade global updates. Prior solutions, such as fixed deadlines, client selection, and layer-wise partial aggregation, alleviate the effect of stragglers, but treat round timing and local workload as static parameters, limiting their effectiveness under strict time constraints. We propose ADEL-FL, a novel framework that jointly optimizes per-round deadlines and user-specific batch sizes for layer-wise aggregation. Our approach formulates a constrained optimization problem minimizing the expected L2 distance to the global optimum under total training time and global rounds. We provide a convergence analysis under exponential compute models and prove that ADEL-FL yields unbiased updates with bounded variance. Extensive experiments demonstrate that ADEL-FL outperforms alternative methods in both convergence rate and final accuracy under heterogeneous conditions.