FedIDM: Achieving Fast and Stable Convergence in Byzantine Federated Learning through Iterative Distribution Matching

TL;DR

FedIDM introduces a distribution matching approach in federated learning to enhance convergence speed and stability while effectively filtering malicious clients, even under severe Byzantine attacks.

Contribution

This work presents FedIDM, a novel method combining distribution matching and robust aggregation to improve Byzantine-robust federated learning.

Findings

FedIDM achieves faster and more stable convergence compared to existing methods.

It maintains acceptable model utility under severe Byzantine attacks.

FedIDM effectively filters malicious clients using distribution matching.

Abstract

Most existing Byzantine-robust federated learning (FL) methods suffer from slow and unstable convergence. Moreover, when handling a substantial proportion of colluded malicious clients, achieving robustness typically entails compromising model utility. To address these issues, this work introduces FedIDM, which employs distribution matching to construct trustworthy condensed data for identifying and filtering abnormal clients. FedIDM consists of two main components: (1) attack-tolerant condensed data generation, and (2) robust aggregation with negative contribution-based rejection. These components exclude local updates that (1) deviate from the update direction derived from condensed data, or (2) cause a significant loss on the condensed dataset. Comprehensive evaluations on three benchmark datasets demonstrate that FedIDM achieves fast and stable convergence while maintaining acceptable model utility, under multiple state-of-the-art Byzantine attacks involving a large number of malicious clients.