BeDKD: Backdoor Defense Based on Directional Mapping Module and Adversarial Knowledge Distillation

TL;DR

BeDKD introduces a novel backdoor defense method that effectively reduces attack success rates by using a directional mapping module and adversarial knowledge distillation, even with limited clean data.

Contribution

The paper proposes BeDKD, a new backdoor defense approach combining directional mapping and adversarial knowledge distillation to improve detection and mitigation with minimal data.

Findings

Reduces attack success rate by 98% on multiple datasets

Outperforms existing state-of-the-art defenses

Maintains high clean accuracy

Abstract

Although existing backdoor defenses have gained success in mitigating backdoor attacks, they still face substantial challenges. In particular, most of them rely on large amounts of clean data to weaken the backdoor mapping but generally struggle with residual trigger effects, resulting in persistently high attack success rates (ASR). Therefore, in this paper, we propose a novel \textbf{B}ackdoor d\textbf{e}fense method based on \textbf{D}irectional mapping module and adversarial \textbf{K}nowledge \textbf{D}istillation (BeDKD), which balances the trade-off between defense effectiveness and model performance using a small amount of clean and poisoned data. We first introduce a directional mapping module to identify poisoned data, which destroys clean mapping while keeping backdoor mapping on a small set of flipped clean data. Then, the adversarial knowledge distillation is designed to reinforce clean mapping and suppress backdoor mapping through a cycle iteration mechanism between trust and punish distillations using clean and identified poisoned data. We conduct experiments to mitigate mainstream attacks on three datasets, and experimental results demonstrate that BeDKD surpasses the state-of-the-art defenses and reduces the ASR by 98$\%$ without significantly reducing the CACC. Our code are available in https://github.com/CAU-ISS-Lab/Backdoor-Attack-Defense-LLMs/tree/main/BeDKD.