Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learning

TL;DR

This paper introduces Fed-DPRoC, a unified federated learning framework that simultaneously achieves differential privacy, Byzantine robustness, and communication efficiency through robust-compatible compression techniques.

Contribution

It presents a novel framework and instantiates it as RobAJoL, integrating Johnson-Lindenstrauss-based compression with robust averaging, backed by theoretical analysis and empirical validation.

Findings

RobAJoL maintains robustness guarantees while reducing communication overhead.

Empirical results show RobAJoL outperforms existing methods under Byzantine attacks.

Theoretical analysis confirms compatibility of JL transform with robust averaging.

Abstract

We propose Fed-DPRoC, a novel federated learning framework designed to jointly provide differential privacy (DP), Byzantine robustness, and communication efficiency. Central to our approach is the concept of robust-compatible compression, which allows reducing the bi-directional communication overhead without undermining the robustness of the aggregation. We instantiate our framework as RobAJoL, which integrates the Johnson-Lindenstrauss (JL)-based compression mechanism with robust averaging for robustness. Our theoretical analysis establishes the compatibility of JL transform with robust averaging, ensuring that RobAJoL maintains robustness guarantees, satisfies DP, and substantially reduces communication overhead. We further present simulation results on CIFAR-10, Fashion MNIST, and FEMNIST, validating our theoretical claims. We compare RobAJoL with a state-of-the-art communication-efficient and robust FL scheme augmented with DP for a fair comparison, demonstrating that RobAJoL outperforms existing methods in terms of robustness and utility under different Byzantine attacks.