FedCliP: Federated Learning with Client Pruning

TL;DR

FedCliP is a federated learning framework that enhances communication efficiency by adaptively pruning clients based on their contribution potential, achieving significant communication savings with minimal accuracy loss.

Contribution

This paper introduces FedCliP, a novel client pruning method with an adaptive contribution score, improving communication efficiency in federated learning under diverse data distributions.

Findings

FedCliP saves up to 70% communication with 0.2% accuracy loss on MNIST.

Achieves 50% and 15% communication reduction on FMNIST and CIFAR-10.

Outperforms state-of-the-art FL frameworks in communication efficiency.

Abstract

The prevalent communication efficient federated learning (FL) frameworks usually take advantages of model gradient compression or model distillation. However, the unbalanced local data distributions (either in quantity or quality) of participating clients, contributing non-equivalently to the global model training, still pose a big challenge to these works. In this paper, we propose FedCliP, a novel communication efficient FL framework that allows faster model training, by adaptively learning which clients should remain active for further model training and pruning those who should be inactive with less potential contributions. We also introduce an alternative optimization method with a newly defined contribution score measure to facilitate active and inactive client determination. We empirically evaluate the communication efficiency of FL frameworks with extensive experiments on three benchmark datasets under both IID and non-IID settings. Numerical results demonstrate the outperformance of the porposed FedCliP framework over state-of-the-art FL frameworks, i.e., FedCliP can save 70% of communication overhead with only 0.2% accuracy loss on MNIST datasets, and save 50% and 15% of communication overheads with less than 1% accuracy loss on FMNIST and CIFAR-10 datasets, respectively.