One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning

TL;DR

This paper introduces PRoBit+, a novel model aggregation algorithm for federated learning that enhances communication efficiency, robustness against Byzantine attacks, and privacy preservation, with theoretical guarantees and empirical validation.

Contribution

The paper proposes PRoBit+, a one-bit stochastic quantization-based aggregation method that improves robustness, privacy, and stability in personalized federated learning with theoretical analysis and extensive experiments.

Findings

PRoBit+ achieves superior Byzantine robustness compared to existing schemes.

PRoBit+ maintains near full-precision performance under privacy constraints.

Theoretical analysis confirms differential privacy and convergence guarantees.

Abstract

As the scale of federated learning (FL) systems expands, their inherent performance limitations like communication overhead, Byzantine vulnerability, and privacy leakage have become increasingly critical. This paper considers a personalized FL framework based on model regularization, and proposes a model aggregation algorithm named PRoBit+ to concurrently overcome these limitations. PRoBit+ employs one-bit stochastic quantization and maximum likelihood estimation for parameter aggregation, and dynamically adjusts the step size of parameter updates, improving training stability of deep neural networks under low communication overhead and heterogeneous data distributions. PRoBit+'s statistical analysis is then conducted and its Byzantine robustness is proved. The $(\epsilon,0)$-differential privacy and a convergence upper bound of the PRoBit+ based FL are also theoretically established in heterogeneous contexts. The analysis illustrates the trade-off among transmission accuracy, security guarantees, and convergence rates, and also indicates that the performance degradation caused by transmission errors and privacy protection can be progressively eliminated at a rate of $\mathcal{O}(1/M)$ as the number of uploading clients $M$ increases. Comprehensive numerical experiments are conducted to assess PRoBit+ in comparison to benchmark methods across different Byzantine attacks and varying proportions of malicious clients. The experimental results demonstrate that PRoBit+ exhibits improved Byzantine robustness over existing bit-based transmission schemes, minimal performance degradation related to privacy protection, and nearly identical performance to full-precision FedAvg in a secure environment.