Divergence-Based Adaptive Aggregation for Byzantine Robust Federated Learning

TL;DR

This paper introduces DRAG and BR-DRAG frameworks for federated learning that effectively mitigate client drifts and Byzantine attacks, ensuring faster convergence and robustness in heterogeneous and adversarial environments.

Contribution

The paper proposes novel divergence-based adaptive aggregation methods, DRAG and BR-DRAG, that improve robustness and convergence in federated learning under data heterogeneity and malicious attacks.

Findings

DRAG outperforms existing methods in handling client drifts.

BR-DRAG maintains robustness against diverse Byzantine attacks.

Both frameworks achieve fast convergence in non-convex models.

Abstract

Inherent client drifts caused by data heterogeneity, as well as vulnerability to Byzantine attacks within the system, hinder effective model training and convergence in federated learning (FL). This paper presents two new frameworks, named DiveRgence-based Adaptive aGgregation (DRAG) and Byzantine-Resilient DRAG (BR-DRAG), to mitigate client drifts and resist attacks while expediting training. DRAG designs a reference direction and a metric named divergence of degree to quantify the deviation of local updates. Accordingly, each worker can align its local update via linear calibration without extra communication cost. BR-DRAG refines DRAG under Byzantine attacks by maintaining a vetted root dataset at the server to produce trusted reference directions. The workers' updates can be then calibrated to mitigate divergence caused by malicious attacks. We analytically prove that DRAG and BR-DRAG achieve fast convergence for non-convex models under partial worker participation, data heterogeneity, and Byzantine attacks. Experiments validate the effectiveness of DRAG and its superior performance over state-of-the-art methods in handling client drifts, and highlight the robustness of BR-DRAG in maintaining resilience against data heterogeneity and diverse Byzantine attacks.