Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data

TL;DR

FedDUMAP is a federated learning framework that enhances training efficiency and accuracy by leveraging shared server data, dynamic updates, adaptive momentum, and layer-specific pruning, outperforming baseline methods significantly.

Contribution

The paper introduces FedDUMAP, a novel federated learning framework combining shared server data, dynamic updates, adaptive momentum, and layer-adaptive pruning for improved efficiency and accuracy.

Findings

Up to 16.9 times faster training.

Up to 20.4% higher accuracy.

Up to 62.6% reduction in computational cost.

Abstract

Despite achieving remarkable performance, Federated Learning (FL) encounters two important problems, i.e., low training efficiency and limited computational resources. In this paper, we propose a new FL framework, i.e., FedDUMAP, with three original contributions, to leverage the shared insensitive data on the server in addition to the distributed data in edge devices so as to efficiently train a global model. First, we propose a simple dynamic server update algorithm, which takes advantage of the shared insensitive data on the server while dynamically adjusting the update steps on the server in order to speed up the convergence and improve the accuracy. Second, we propose an adaptive optimization method with the dynamic server update algorithm to exploit the global momentum on the server and each local device for superior accuracy. Third, we develop a layer-adaptive model pruning method to carry out specific pruning operations, which is adapted to the diverse features of each layer so as to attain an excellent trade-off between effectiveness and efficiency. Our proposed FL model, FedDUMAP, combines the three original techniques and has a significantly better performance compared with baseline approaches in terms of efficiency (up to 16.9 times faster), accuracy (up to 20.4% higher), and computational cost (up to 62.6% smaller).