Defending the Edge: Representative-Attention Defense against Backdoor Attacks in Federated Learning

TL;DR

This paper proposes FeRA, an attention-based defense mechanism for federated learning that detects backdoor attacks by analyzing consistency in representation space rather than traditional anomaly detection methods.

Contribution

FeRA introduces a novel attention-driven approach that identifies malicious clients through consistency analysis and norm-inflation detection, addressing limitations of existing anomaly-based defenses.

Findings

FeRA achieves about 1.67% backdoor accuracy under non-IID settings.

It maintains high clean accuracy while effectively mitigating backdoor attacks.

Extensive tests across datasets and models confirm its superior performance.

Abstract

Federated learning (FL) remains highly vulnerable to adaptive backdoor attacks that preserve stealth by closely imitating benign update statistics. Existing defenses predominantly rely on anomaly detection in parameter or gradient space, overlooking behavioral constraints that backdoor attacks must satisfy to ensure reliable trigger activation. These anomaly-centric methods fail against adaptive attacks that normalize update magnitudes and mimic benign statistical patterns while preserving backdoor functionality, creating a fundamental detection gap. To address this limitation, this paper introduces FeRA (Federated Representative Attention) -- a novel attention-driven defense that shifts the detection paradigm from anomaly-centric to consistency-centric analysis. FeRA exploits the intrinsic need for backdoor persistence across training rounds, identifying malicious clients through suppressed representation-space variance, an orthogonal property to traditional magnitude-based statistics. The framework conducts multi-dimensional behavioral analysis combining spectral and spatial attention, directional alignment, mutual similarity, and norm inflation across two complementary detection mechanisms: consistency analysis and norm-inflation detection. Through this mechanism, FeRA isolates malicious clients that exhibit low-variance consistency or magnitude amplification. Extensive evaluation across six datasets, nine attacks, and three model architectures under both Independent and Identically Distributed (IID) and non-IID settings confirm FeRA achieves superior backdoor mitigation. Under different non-IID settings, FeRA achieved the lowest average Backdoor Accuracy (BA), about 1.67% while maintaining high clean accuracy compared to other state-of-the-art defenses. The code is available at https://github.com/Peatech/FeRA_defense.git.