Private and Communication-Efficient Federated Learning based on Differentially Private Sketches

TL;DR

This paper introduces DPSFL, a federated learning approach that combines differential privacy with gradient sketching to enhance privacy and communication efficiency, and proposes an adaptive clipping strategy to improve model performance.

Contribution

The paper presents DPSFL, a novel federated learning method using differentially private sketches, and introduces DPSFL-AC with adaptive clipping to mitigate bias from gradient clipping.

Findings

DPSFL improves communication efficiency and privacy preservation.

DPSFL-AC reduces bias and enhances model accuracy.

Experimental results outperform existing methods in privacy, efficiency, and accuracy.

Abstract

Federated learning (FL) faces two primary challenges: the risk of privacy leakage due to parameter sharing and communication inefficiencies. To address these challenges, we propose DPSFL, a federated learning method that utilizes differentially private sketches. DPSFL compresses the local gradients of each client using a count sketch, thereby improving communication efficiency, while adding noise to the sketches to ensure differential privacy (DP). We provide a theoretical analysis of privacy and convergence for the proposed method. Gradient clipping is essential in DP learning to limit sensitivity and constrain the addition of noise. However, clipping introduces bias into the gradients, negatively impacting FL performance. To mitigate the impact of clipping, we propose an enhanced method, DPSFL-AC, which employs an adaptive clipping strategy. Experimental comparisons with existing techniques demonstrate the superiority of our methods concerning privacy preservation, communication efficiency, and model accuracy.