Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning

TL;DR

This paper introduces Load-aware Tram-FL, a decentralized federated learning method that optimizes training scheduling by balancing computational and communication loads, significantly reducing training time and improving convergence.

Contribution

It formulates a load-aware training scheduling mechanism for decentralized federated learning, decomposing a complex optimization problem into node-wise subproblems for practical implementation.

Findings

Reduces total training time in simulations on MNIST and CIFAR-10.

Accelerates convergence compared to baseline methods.

Balances data utilization under non-IID distributions.

Abstract

This paper proposes Load-aware Tram-FL, an extension of Tram-FL that introduces a training scheduling mechanism to minimize total training time in decentralized federated learning by accounting for both computational and communication loads. The scheduling problem is formulated as a global optimization task, which-though intractable in its original form-is made solvable by decomposing it into node-wise subproblems. To promote balanced data utilization under non-IID distributions, a variance constraint is introduced, while the overall training latency, including both computation and communication costs, is minimized through the objective function. Simulation results on MNIST and CIFAR-10 demonstrate that Load-aware Tram-FL significantly reduces training time and accelerates convergence compared to baseline methods.