Asynchronous Byzantine Federated Learning

TL;DR

This paper introduces a novel asynchronous Byzantine-resilient federated learning algorithm that improves training speed and robustness against attacks without requiring an auxiliary dataset or being delayed by slow clients.

Contribution

It presents one of the first asynchronous Byzantine fault-tolerant FL algorithms that does not rely on auxiliary datasets and handles stragglers effectively.

Findings

Faster training compared to synchronous FL

Higher accuracy under attack scenarios

Robustness against gradient inversion, perturbation, and backdoor attacks

Abstract

Federated learning (FL) enables a set of geographically distributed clients to collectively train a model through a server. Classically, the training process is synchronous, but can be made asynchronous to maintain its speed in presence of slow clients and in heterogeneous networks. The vast majority of Byzantine fault-tolerant FL systems however rely on a synchronous training process. Our solution is one of the first Byzantine-resilient and asynchronous FL algorithms that does not require an auxiliary server dataset and is not delayed by stragglers, which are shortcomings of previous works. Intuitively, the server in our solution waits to receive a minimum number of updates from clients on its latest model to safely update it, and is later able to safely leverage the updates that late clients might send. We compare the performance of our solution with state-of-the-art algorithms on both image and text datasets under gradient inversion, perturbation, and backdoor attacks. Our results indicate that our solution trains a model faster than previous synchronous FL solution, and maintains a higher accuracy, up to 1.54x and up to 1.75x for perturbation and gradient inversion attacks respectively, in the presence of Byzantine clients than previous asynchronous FL solutions.