Byzantine-Robust Federated Learning over Ring-All-Reduce Distributed Computing

TL;DR

This paper introduces BRACE, a novel Byzantine-robust federated learning algorithm based on ring-all-reduce architecture, ensuring both security against malicious attacks and communication efficiency, with proven convergence and validated experiments.

Contribution

We propose the first Byzantine-robust federated learning algorithm using ring-all-reduce, addressing a key open problem and providing theoretical guarantees and practical validation.

Findings

BRACE achieves Byzantine robustness in RAR-based FL.

BRACE maintains communication efficiency and bandwidth optimality.

Experimental results validate the effectiveness of BRACE against Byzantine attacks.

Abstract

Federated learning (FL) has gained attention as a distributed learning paradigm for its data privacy benefits and accelerated convergence through parallel computation. Traditional FL relies on a server-client (SC) architecture, where a central server coordinates multiple clients to train a global model, but this approach faces scalability challenges due to server communication bottlenecks. To overcome this, the ring-all-reduce (RAR) architecture has been introduced, eliminating the central server and achieving bandwidth optimality. However, the tightly coupled nature of RAR's ring topology exposes it to unique Byzantine attack risks not present in SC-based FL. Despite its potential, designing Byzantine-robust RAR-based FL algorithms remains an open problem. To address this gap, we propose BRACE (Byzantine-robust ring-all-reduce), the first RAR-based FL algorithm to achieve both Byzantine robustness and communication efficiency. We provide theoretical guarantees for the convergence of BRACE under Byzantine attacks, demonstrate its bandwidth efficiency, and validate its practical effectiveness through experiments. Our work offers a foundational understanding of Byzantine-robust RAR-based FL design.