Bayesian Robust Aggregation for Federated Learning

TL;DR

This paper introduces a Bayesian inference-based robust aggregation method for federated learning that effectively defends against adversarial attacks, maintaining simplicity and high performance even with unknown or varying numbers of malicious clients.

Contribution

It proposes an adaptive Bayesian approach for aggregating model updates in federated learning, robust to unknown and varying malicious client presence.

Findings

Achieves state-of-the-art robustness across multiple attack types.

Performs consistently well with static and dynamic malicious client scenarios.

Comparable to specialized methods like Krum in defending against attacks.

Abstract

Federated Learning enables collaborative training of machine learning models on decentralized data. This scheme, however, is vulnerable to adversarial attacks, when some of the clients submit corrupted model updates. In real-world scenarios, the total number of compromised clients is typically unknown, with the extent of attacks potentially varying over time. To address these challenges, we propose an adaptive approach for robust aggregation of model updates based on Bayesian inference. The mean update is defined by the maximum of the likelihood marginalized over probabilities of each client to be `honest'. As a result, the method shares the simplicity of the classical average estimators (e.g., sample mean or geometric median), being independent of the number of compromised clients. At the same time, it is as effective against attacks as methods specifically tailored to Federated Learning, such as Krum. We compare our approach with other aggregation schemes in federated setting on three benchmark image classification data sets. The proposed method consistently achieves state-of-the-art performance across various attack types with static and varying number of malicious clients.