Defending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization

TL;DR

This paper introduces a robust federated learning method using consensus-based bi-level optimization, providing theoretical convergence guarantees and demonstrating effectiveness against adversarial attacks in practical decentralized settings.

Contribution

It develops CB$^2$O, a new multi-particle optimization method with proven robustness, and extends it to FedCB$^2$O for clustered federated learning, addressing real-world adversarial challenges.

Findings

CB$^2$O converges globally in mean-field law under adversarial conditions.

FedCB$^2$O effectively defends against label-flipping attacks.

The proposed methods enhance robustness in decentralized federated learning environments.

Abstract

Adversarial attacks pose significant challenges in many machine learning applications, particularly in the setting of distributed training and federated learning, where malicious agents seek to corrupt the training process with the goal of jeopardizing and compromising the performance and reliability of the final models. In this paper, we address the problem of robust federated learning in the presence of such attacks by formulating the training task as a bi-level optimization problem. We conduct a theoretical analysis of the resilience of consensus-based bi-level optimization (CB$^2$O), an interacting multi-particle metaheuristic optimization method, in adversarial settings. Specifically, we provide a global convergence analysis of CB$^2$O in mean-field law in the presence of malicious agents, demonstrating the robustness of CB$^2$O against a diverse range of attacks. Thereby, we offer insights into how specific hyperparameter choices enable to mitigate adversarial effects. On the practical side, we extend CB$^2$O to the clustered federated learning setting by proposing FedCB$^2$O, a novel interacting multi-particle system, and design a practical algorithm that addresses the demands of real-world applications. Extensive experiments demonstrate the robustness of the FedCB$^2$O algorithm against label-flipping attacks in decentralized clustered federated learning scenarios, showcasing its effectiveness in practical contexts.