On the Geometric Convergence of Byzantine-Resilient Distributed Optimization Algorithms

TL;DR

This paper introduces a general framework for Byzantine-resilient distributed optimization, demonstrating geometric convergence to a near-optimal solution and providing insights into the factors affecting convergence speed and accuracy.

Contribution

It presents a unified algorithmic framework for Byzantine resilience in distributed optimization and analyzes its geometric convergence properties, including convergence rate and conditions.

Findings

Algorithms converge geometrically to a ball around the optimal solution.

Approximate consensus is achieved rapidly under minimal conditions.

The convergence region and rate depend on step-size and function properties.

Abstract

The problem of designing distributed optimization algorithms that are resilient to Byzantine adversaries has received significant attention. For the Byzantine-resilient distributed optimization problem, the goal is to (approximately) minimize the average of the local cost functions held by the regular (non adversarial) agents in the network. In this paper, we provide a general algorithmic framework for Byzantine-resilient distributed optimization which includes some state-of-the-art algorithms as special cases. We analyze the convergence of algorithms within the framework, and derive a geometric rate of convergence of all regular agents to a ball around the optimal solution (whose size we characterize). Furthermore, we show that approximate consensus can be achieved geometrically fast under some minimal conditions. Our analysis provides insights into the relationship among the convergence region, distance between regular agents' values, step-size, and properties of the agents' functions for Byzantine-resilient distributed optimization.