Sparse Personalized Federated Learning

TL;DR

This paper introduces FedMac, a sparse personalized federated learning method that enhances model performance and reduces communication costs by incorporating sparsity constraints and correlation measures, with proven convergence and superior accuracy.

Contribution

The paper proposes FedMac, a novel sparse personalized federated learning scheme that improves performance and efficiency through correlation-based sparsity constraints, with theoretical convergence guarantees.

Findings

FedMac achieves high accuracy on multiple datasets under non-i.i.d. conditions.

The method reduces communication and computational loads compared to non-sparse FL.

Convergence analysis confirms that sparsity constraints do not hinder model convergence.

Abstract

Federated Learning (FL) is a collaborative machine learning technique to train a global model without obtaining clients' private data. The main challenges in FL are statistical diversity among clients, limited computing capability among clients' equipments, and the excessive communication overhead between the server and clients. To address these challenges, we propose a novel sparse personalized federated learning scheme via maximizing correlation (FedMac). By incorporating an approximated L1-norm and the correlation between client models and global model into standard FL loss function, the performance on statistical diversity data is improved and the communicational and computational loads required in the network are reduced compared with non-sparse FL. Convergence analysis shows that the sparse constraints in FedMac do not affect the convergence rate of the global model, and theoretical results show that FedMac can achieve good sparse personalization, which is better than the personalized methods based on L2-norm. Experimentally, we demonstrate the benefits of this sparse personalization architecture compared with the state-of-the-art personalization methods (e.g. FedMac respectively achieves 98.95%, 99.37%, 90.90%, 89.06% and 73.52% accuracy on the MNIST, FMNIST, CIFAR-100, Synthetic and CINIC-10 datasets under non-i.i.d. variants).