TT-Prune: Joint Model Pruning and Resource Allocation for Communication-efficient Time-triggered Federated Learning

TL;DR

This paper proposes a joint model pruning and resource allocation approach for time-triggered federated learning to reduce communication costs and latency, ensuring efficient and privacy-preserving distributed training.

Contribution

It introduces adaptive model pruning combined with bandwidth optimization in TT-Fed, providing a novel method to minimize training loss and communication overhead.

Findings

Model pruning reduces communication cost by 40%.

Joint optimization maintains model performance while decreasing latency.

Closed-form solutions for bandwidth and pruning ratio are derived.

Abstract

Federated learning (FL) offers new opportunities in machine learning, particularly in addressing data privacy concerns. In contrast to conventional event-based federated learning, time-triggered federated learning (TT-Fed), as a general form of both asynchronous and synchronous FL, clusters users into different tiers based on fixed time intervals. However, the FL network consists of a growing number of user devices with limited wireless bandwidth, consequently magnifying issues such as stragglers and communication overhead. In this paper, we introduce adaptive model pruning to wireless TT-Fed systems and study the problem of jointly optimizing the pruning ratio and bandwidth allocation to minimize the training loss while ensuring minimal learning latency. To answer this question, we perform convergence analysis on the gradient l_2 norm of the TT-Fed model based on model pruning. Based on the obtained convergence upper bound, a joint optimization problem of pruning ratio and wireless bandwidth is formulated to minimize the model training loss under a given delay threshold. Then, we derive closed-form solutions for wireless bandwidth and pruning ratio using Karush-Kuhn-Tucker(KKT) conditions. The simulation results show that model pruning could reduce the communication cost by 40% while maintaining the model performance at the same level.