Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

TL;DR

This paper introduces a multi-stage, privacy-preserving, Byzantine-resilient federated learning scheme that effectively handles heterogeneous data, combining verifiable secret sharing, secure aggregation, and private information retrieval to improve robustness and scalability.

Contribution

It presents a novel co-designed approach integrating verifiable secret sharing, secure aggregation, and private information retrieval for privacy and Byzantine resilience in heterogeneous federated learning.

Findings

Outperforms previous techniques in robustness against attacks.

Achieves information-theoretic privacy guarantees.

Reduces communication costs with zero-order estimation methods.

Abstract

Ensuring resilience to Byzantine clients while maintaining the privacy of the clients' data is a fundamental challenge in federated learning (FL). When the clients' data is homogeneous, suitable countermeasures were studied from an information-theoretic perspective utilizing secure aggregation techniques while ensuring robust aggregation of the clients' gradients. However, the countermeasures used fail when the clients' data is heterogeneous. Suitable pre-processing techniques, such as nearest neighbor mixing, were recently shown to enhance the performance of those countermeasures in the heterogeneous setting. Nevertheless, those pre-processing techniques cannot be applied with the introduced privacy-preserving mechanisms. We propose a multi-stage method encompassing a careful co-design of verifiable secret sharing, secure aggregation, and a tailored symmetric private information retrieval scheme to achieve information-theoretic privacy guarantees and Byzantine resilience under data heterogeneity. We evaluate the effectiveness of our scheme on a variety of attacks and show how it outperforms the previously known techniques. Since the communication overhead of secure aggregation is non-negligible, we investigate the interplay with zero-order estimation methods that reduce the communication cost in state-of-the-art FL tasks and thereby make private aggregation scalable.