Federated Learning with Unreliable Clients: Performance Analysis and Mechanism Design

TL;DR

This paper analyzes the impact of unreliable clients in federated learning, providing theoretical bounds, and proposes DeepSA, a neural network-based secure aggregation method to improve robustness and performance.

Contribution

It models unreliable client behaviors in federated learning, derives convergence bounds, and introduces DeepSA, a novel neural network-based defense mechanism.

Findings

Theoretical convergence bounds depend on the number of local iterations.

DeepSA effectively mitigates the impact of unreliable clients.

Experimental results outperform existing defenses.

Abstract

Owing to the low communication costs and privacy-promoting capabilities, Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients. However, with the distributed architecture, low quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training. In this paper, we model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk. Specifically, we first investigate the impact on the models caused by unreliable clients by deriving a convergence upper bound on the loss function based on the gradient descent updates. Our theoretical bounds reveal that with a fixed amount of total computational resources, there exists an optimal number of local training iterations in terms of convergence performance. We further design a novel defensive mechanism, named deep neural network based secure aggregation (DeepSA). Our experimental results validate our theoretical analysis. In addition, the effectiveness of DeepSA is verified by comparing with other state-of-the-art defensive mechanisms.