A Seesaw Model Attack Algorithm for Distributed Learning

TL;DR

This paper introduces the seesaw attack, a novel and more effective Byzantine attack method for distributed learning systems, highlighting vulnerabilities in existing gradient aggregation rules and emphasizing the need for robust defenses.

Contribution

The paper proposes the seesaw attack algorithm, demonstrating its superiority over finite-norm attacks in compromising distributed learning models.

Findings

Seesaw attack outperforms finite-norm attacks in effectiveness.

The attack successfully compromises various gradient aggregation rules.

Experimental results confirm the attack's robustness across different scenarios.

Abstract

We investigate the Byzantine attack problem within the context of model training in distributed learning systems. While ensuring the convergence of current model training processes, common solvers (e.g. SGD, Adam, RMSProp, etc.) can be easily compromised by malicious nodes in these systems. Consequently, the training process may either converge slowly or even diverge. To develop effective secure distributed learning solvers, it is crucial to first examine attack methods to assess the robustness of these solvers. In this work, we contribute to the design of attack strategies by initially highlighting the limitations of finite-norm attacks. We then introduce the seesaw attack, which has been demonstrated to be more effective than the finite-norm attack. Through numerical experiments, we evaluate the efficacy of the seesaw attack across various gradient aggregation rules.