FedSECA: Sign Election and Coordinate-wise Aggregation of Gradients for Byzantine Tolerant Federated Learning

TL;DR

FedSECA introduces a novel robust aggregation method for federated learning that enhances resistance against Byzantine attacks by using sign election and coordinate-wise aggregation based on client agreement metrics.

Contribution

The paper proposes FedSECA, a new Byzantine-tolerant federated learning method combining sign election and coordinate-wise aggregation with a novel concordance ratio metric.

Findings

FedSECA outperforms 10 existing robust aggregators under 7 Byzantine attacks.

Existing aggregators fail against certain high-strength attacks, while FedSECA maintains robustness.

FedSECA demonstrates improved accuracy and resilience across multiple datasets and architectures.

Abstract

One of the most common defense strategies against Byzantine clients in federated learning (FL) is to employ a robust aggregator mechanism that makes the training more resilient. While many existing Byzantine robust aggregators provide theoretical convergence guarantees and are empirically effective against certain categories of attacks, we observe that certain high-strength attacks can subvert the robust aggregator and collapse the training. To overcome this limitation, we propose a method called FedSECA for robust Sign Election and Coordinate-wise Aggregation of gradients in FL that is less susceptible to malicious updates by an omniscient attacker. The proposed method has two main components. The Concordance Ratio Induced Sign Election(CRISE) module determines the consensus direction (elected sign) for each individual parameter gradient through a weighted voting strategy. The client weights are assigned based on a novel metric called concordance ratio, which quantifies the degree of sign agreement between the client gradient updates. Based on the elected sign, a Robust Coordinate-wise Aggregation(RoCA) strategy is employed, where variance-reduced sparse gradients are aggregated only if they are in alignment with the corresponding elected sign. We compare our proposed FedSECA method against 10 robust aggregators under 7 Byzantine attacks on 3 datasets and architectures. The results show that existing robust aggregators fail for at least some attacks, while FedSECA exhibits better robustness. Code - https://github.com/JosephGeoBenjamin/FedSECA-ByzantineTolerance