Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation

TL;DR

This paper introduces LASA, a novel layer-adaptive sparsified aggregation method that enhances Byzantine resilience in federated learning by reducing malicious influence and adaptively filtering updates across layers.

Contribution

LASA combines pre-aggregation sparsification with layer-wise adaptive filtering to improve robustness against Byzantine attacks in federated learning, especially in non-IID settings.

Findings

LASA significantly improves robustness in non-IID federated learning scenarios.

LASA outperforms existing methods in defending against Byzantine attacks.

Theoretical analysis confirms LASA's resilience and robustness advantages.

Abstract

Federated Learning (FL) enables multiple clients to collaboratively train a model without sharing their local data. Yet the FL system is vulnerable to well-designed Byzantine attacks, which aim to disrupt the model training process by uploading malicious model updates. Existing robust aggregation rule-based defense methods overlook the diversity of magnitude and direction across different layers of the model updates, resulting in limited robustness performance, particularly in non-IID settings. To address these challenges, we propose the Layer-Adaptive Sparsified Model Aggregation (LASA) approach, which combines pre-aggregation sparsification with layer-wise adaptive aggregation to improve robustness. Specifically, LASA includes a pre-aggregation sparsification module that sparsifies updates from each client before aggregation, reducing the impact of malicious parameters and minimizing the interference from less important parameters for the subsequent filtering process. Based on sparsified updates, a layer-wise adaptive filter then adaptively selects benign layers using both magnitude and direction metrics across all clients for aggregation. We provide the detailed theoretical robustness analysis of LASA and the resilience analysis for the FL integrated with LASA. Extensive experiments are conducted on various IID and non-IID datasets. The numerical results demonstrate the effectiveness of LASA. Code is available at \url{https://github.com/JiiahaoXU/LASA}.