Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learning

TL;DR

This paper investigates how different network topologies affect the robustness of decentralized federated learning against Byzantine nodes, revealing that current aggregation methods lack resilience in complex, large-scale networks.

Contribution

It provides empirical analysis of Byzantine-robust aggregation methods across various network topologies, highlighting their limitations in non-fully connected, large-scale networks.

Findings

Current Byzantine-robust methods are ineffective in large, complex networks.

Network topology significantly impacts Byzantine resilience.

Development of topology-aware aggregation schemes is necessary.

Abstract

Federated learning (FL) enables a collaborative environment for training machine learning models without sharing training data between users. This is typically achieved by aggregating model gradients on a central server. Decentralized federated learning is a rising paradigm that enables users to collaboratively train machine learning models in a peer-to-peer manner, without the need for a central aggregation server. However, before applying decentralized FL in real-world use training environments, nodes that deviate from the FL process (Byzantine nodes) must be considered when selecting an aggregation function. Recent research has focused on Byzantine-robust aggregation for client-server or fully connected networks, but has not yet evaluated such aggregation schemes for complex topologies possible with decentralized FL. Thus, the need for empirical evidence of Byzantine robustness in differing network topologies is evident. This work investigates the effects of state-of-the-art Byzantine-robust aggregation methods in complex, large-scale network structures. We find that state-of-the-art Byzantine robust aggregation strategies are not resilient within large non-fully connected networks. As such, our findings point the field towards the development of topology-aware aggregation schemes, especially necessary within the context of large scale real-world deployment.