Sparse Decentralized Federated Learning

TL;DR

This paper introduces Sparse DFL with the CEPS algorithm, combining sparsity, one-bit compressive sensing, and differential privacy to enhance efficiency, stability, and trustworthiness in decentralized federated learning.

Contribution

It proposes a novel sparsity-constrained algorithm, CEPS, with theoretical guarantees for convergence and privacy, addressing key challenges in decentralized federated learning.

Findings

Improved communication efficiency through one-bit compressive sensing.

Enhanced privacy and trustworthiness via integrated differential privacy.

Validated effectiveness through numerical experiments.

Abstract

Decentralized Federated Learning (DFL) enables collaborative model training without a central server but faces challenges in efficiency, stability, and trustworthiness due to communication and computational limitations among distributed nodes. To address these critical issues, we introduce a sparsity constraint on the shared model, leading to Sparse DFL (SDFL), and propose a novel algorithm, CEPS. The sparsity constraint facilitates the use of one-bit compressive sensing to transmit one-bit information between partially selected neighbour nodes at specific steps, thereby significantly improving communication efficiency. Moreover, we integrate differential privacy into the algorithm to ensure privacy preservation and bolster the trustworthiness of the learning process. Furthermore, CEPS is underpinned by theoretical guarantees regarding both convergence and privacy. Numerical experiments validate the effectiveness of the proposed algorithm in improving communication and computation efficiency while maintaining a high level of trustworthiness.