Phocas: dimensional Byzantine-resilient stochastic gradient descent

TL;DR

Phocas introduces a new Byzantine-resilient aggregation method for distributed SGD that effectively withstands arbitrary malicious attacks, improving robustness and performance in distributed machine learning.

Contribution

The paper presents a novel aggregation rule for distributed SGD that guarantees Byzantine resilience under general attack models, with proven theoretical robustness.

Findings

Outperforms existing methods in realistic attack scenarios

Proven Byzantine resilience of the proposed aggregation rule

Empirical results demonstrate improved robustness and accuracy

Abstract

We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience of the proposed aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios.