Dim-Krum: Backdoor-Resistant Federated Learning for NLP with Dimension-wise Krum-Based Aggregation

TL;DR

This paper introduces Dim-Krum, a new federated learning aggregation method designed to resist backdoor attacks in NLP, addressing the unique challenges posed by NLP backdoors that are harder to defend against than in computer vision.

Contribution

The paper provides a theoretical analysis of NLP backdoor attack detection and proposes Dim-Krum, a dimension-wise Krum-based aggregation method tailored for NLP backdoor resistance.

Findings

Dim-Krum effectively reduces backdoor risks in NLP federated learning.

NLP backdoors tend to have small relative strengths, making them harder to detect.

Experimental results show Dim-Krum outperforms existing robust aggregation methods.

Abstract

Despite the potential of federated learning, it is known to be vulnerable to backdoor attacks. Many robust federated aggregation methods are proposed to reduce the potential backdoor risk. However, they are mainly validated in the CV field. In this paper, we find that NLP backdoors are hard to defend against than CV, and we provide a theoretical analysis that the malicious update detection error probabilities are determined by the relative backdoor strengths. NLP attacks tend to have small relative backdoor strengths, which may result in the failure of robust federated aggregation methods for NLP attacks. Inspired by the theoretical results, we can choose some dimensions with higher backdoor strengths to settle this issue. We propose a novel federated aggregation algorithm, Dim-Krum, for NLP tasks, and experimental results validate its effectiveness.