Joint Model Pruning and Resource Allocation for Wireless Time-triggered Federated Learning

TL;DR

This paper proposes a joint optimization approach for model pruning and bandwidth allocation in wireless time-triggered federated learning, significantly reducing communication costs while maintaining convergence.

Contribution

It introduces a convergence analysis and closed-form solutions for jointly optimizing pruning ratio and bandwidth in wireless federated learning.

Findings

40% reduction in communication cost with TT-Prune

Maintains model convergence comparable to non-pruned systems

Provides a theoretical framework with KKT-based solutions

Abstract

Time-triggered federated learning, in contrast to conventional event-based federated learning, organizes users into tiers based on fixed time intervals. However, this network still faces challenges due to a growing number of devices and limited wireless bandwidth, increasing issues like stragglers and communication overhead. In this paper, we apply model pruning to wireless Time-triggered systems and jointly study the problem of optimizing the pruning ratio and bandwidth allocation to minimize training loss under communication latency constraints. To solve this joint optimization problem, we perform a convergence analysis on the gradient $l_2$-norm of the asynchronous multi-tier federated learning (FL) model with adaptive model pruning. The convergence upper bound is derived and a joint optimization problem of pruning ratio and wireless bandwidth is defined to minimize the model training loss under a given communication latency constraint. The closed-form solutions for wireless bandwidth and pruning ratio by using KKT conditions are then formulated. As indicated in the simulation experiments, our proposed TT-Prune demonstrates a 40% reduction in communication cost, compared with the asynchronous multi-tier FL without model pruning, while maintaining the model convergence at the same level.