Byzantine-robust Federated Learning through Spatial-temporal Analysis of Local Model Updates

TL;DR

This paper introduces a novel spatial-temporal analysis approach for federated learning that detects and mitigates Byzantine client attacks, improving robustness against malicious updates while preserving privacy.

Contribution

It presents a clustering-based detection method combined with adaptive learning rate adjustment to enhance Byzantine robustness in federated learning.

Findings

Outperforms existing methods in robustness under malicious attacks

Effective in both cross-silo and cross-device federated learning scenarios

Reduces impact of faulty or malicious client updates

Abstract

Federated Learning (FL) enables multiple distributed clients (e.g., mobile devices) to collaboratively train a centralized model while keeping the training data locally on the client. Compared to traditional centralized machine learning, FL offers many favorable features such as offloading operations which would usually be performed by a central server and reducing risks of serious privacy leakage. However, Byzantine clients that send incorrect or disruptive updates due to system failures or adversarial attacks may disturb the joint learning process, consequently degrading the performance of the resulting model. In this paper, we propose to mitigate these failures and attacks from a spatial-temporal perspective. Specifically, we use a clustering-based method to detect and exclude incorrect updates by leveraging their geometric properties in the parameter space. Moreover, to further handle malicious clients with time-varying behaviors, we propose to adaptively adjust the learning rate according to momentum-based update speculation. Extensive experiments on 4 public datasets demonstrate that our algorithm achieves enhanced robustness comparing to existing methods under both cross-silo and cross-device FL settings with faulty/malicious clients.