DP-SIGNSGD: When Efficiency Meets Privacy and Robustness

TL;DR

DP-SIGNSGD enhances federated learning by combining efficiency, privacy, and robustness, addressing key challenges with improved methods and demonstrating effectiveness on benchmark datasets.

Contribution

The paper introduces DP-SIGNSGD, a novel method that guarantees privacy while maintaining efficiency and robustness in federated learning, along with an error-feedback variant for better accuracy.

Findings

Effective privacy guarantees demonstrated

Maintains efficiency and robustness in FL

Improved accuracy with error-feedback variant

Abstract

Federated learning (FL) has emerged as a promising collaboration paradigm by enabling a multitude of parties to construct a joint model without exposing their private training data. Three main challenges in FL are efficiency, privacy, and robustness. The recently proposed SIGNSGD with majority vote shows a promising direction to deal with efficiency and Byzantine robustness. However, there is no guarantee that SIGNSGD is privacy-preserving. In this paper, we bridge this gap by presenting an improved method called DP-SIGNSGD, which can meet all the aforementioned properties. We further propose an error-feedback variant of DP-SIGNSGD to improve accuracy. Experimental results on benchmark image datasets demonstrate the effectiveness of our proposed methods.