Byzantine-Robust Gossip: Insights from a Dual Approach

TL;DR

This paper introduces a Byzantine-robust decentralized optimization algorithm using a dual approach, providing convergence guarantees and experimental validation to enhance the resilience of distributed learning against malicious devices.

Contribution

It proposes a novel Byzantine-robust algorithm based on the dual approach, offering theoretical convergence guarantees and reinterpretation of existing methods.

Findings

Convergence guarantees in the average consensus subcase

Reinterpretation of existing algorithms within the dual framework

Experimental validation demonstrating method effectiveness

Abstract

Distributed learning has many computational benefits but is vulnerable to attacks from a subset of devices transmitting incorrect information. This paper investigates Byzantine-resilient algorithms in a decentralized setting, where devices communicate directly in a peer-to-peer manner within a communication network. We leverage the so-called dual approach for decentralized optimization and propose a Byzantine-robust algorithm. We provide convergence guarantees in the average consensus subcase, discuss the potential of the dual approach beyond this subcase, and re-interpret existing algorithms using the dual framework. Lastly, we experimentally show the soundness of our method.