Generalized Byzantine-tolerant SGD
Cong Xie, Oluwasanmi Koyejo, Indranil Gupta

TL;DR
This paper introduces three new robust aggregation rules for distributed SGD that withstand Byzantine failures, demonstrating improved resilience and performance in adversarial scenarios.
Contribution
The paper proposes three novel aggregation rules for Byzantine-tolerant distributed SGD and proves their resilience under a general Byzantine failure model.
Findings
Proposed aggregation rules outperform existing methods in realistic scenarios.
The rules are proven to be Byzantine resilient.
Empirical results confirm improved robustness and efficiency.
Abstract
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience properties of these aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDistributed systems and fault tolerance · Privacy-Preserving Technologies in Data · Stochastic Gradient Optimization Techniques
