Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions

TL;DR

This paper introduces Byz-Clip21-SGD2M, a federated learning algorithm that enhances Byzantine robustness and differential privacy under realistic assumptions, with proven convergence and empirical validation.

Contribution

It proposes a novel algorithm combining robust aggregation, double momentum, and clipping, with theoretical convergence guarantees under standard assumptions.

Findings

Achieves high-probability convergence guarantees under standard assumptions.

Recovers state-of-the-art rates without adversaries and improves utility with Byzantine and DP.

Empirically outperforms existing methods on MNIST with CNN and MLP models.

Abstract

Federated Learning (FL) enables heterogeneous clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy. However, gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation. These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework. Existing approaches, however, often rely on unrealistic assumptions such as bounded gradients, require auxiliary server-side datasets, or fail to provide convergence guarantees. We address these limitations by proposing Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping. We prove high-probability convergence guarantees under standard $L$-smoothness and $\sigma$-sub-Gaussian gradient noise assumptions, thereby relaxing conditions that dominate prior work. Our analysis recovers state-of-the-art convergence rates in the absence of adversaries and improves utility guarantees under Byzantine and DP settings. Empirical evaluations on CNN and MLP models trained on MNIST further validate the effectiveness of our approach.