Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learning

TL;DR

This paper introduces two federated learning schemes, ByITFL and LoByITFL, that provide perfect privacy and Byzantine resilience, with trade-offs in communication overhead and trust assumptions, supported by theoretical and experimental validation.

Contribution

It presents the first Byzantine-resilient FL schemes with perfect information-theoretic privacy, balancing privacy, resilience, and communication efficiency.

Findings

ByITFL achieves perfect IT privacy with high communication overhead.

LoByITFL offers Byzantine resilience and IT privacy with lower communication costs.

Both schemes are theoretically guaranteed and experimentally validated.

Abstract

Federated learning (FL) shows great promise in large-scale machine learning but introduces new privacy and security challenges. We propose ByITFL and LoByITFL, two novel FL schemes that enhance resilience against Byzantine users while keeping the users' data private from eavesdroppers. To ensure privacy and Byzantine resilience, our schemes build on having a small representative dataset available to the federator and crafting a discriminator function allowing the mitigation of corrupt users' contributions. ByITFL employs Lagrange coded computing and re-randomization, making it the first Byzantine-resilient FL scheme with perfect Information-Theoretic (IT) privacy, though at the cost of a significant communication overhead. LoByITFL, on the other hand, achieves Byzantine resilience and IT privacy at a significantly reduced communication cost, but requires a Trusted Third Party, used only in a one-time initialization phase before training. We provide theoretical guarantees on privacy and Byzantine resilience, along with convergence guarantees and experimental results validating our findings.