Adversarial Backdoor Defense in CLIP

TL;DR

This paper introduces Adversarial Backdoor Defense (ABD), a novel data augmentation method that enhances CLIP's robustness against backdoor attacks by aligning features with adversarial examples, significantly reducing attack success rates.

Contribution

The paper proposes ABD, a new adversarial data augmentation strategy that effectively defends multimodal models like CLIP against backdoor attacks, outperforming existing methods.

Findings

ABD reduces attack success rates significantly across multiple backdoor attack types.

ABD maintains high clean accuracy with minimal performance loss.

ABD outperforms the state-of-the-art defense method, CleanCLIP.

Abstract

Multimodal contrastive pretraining, exemplified by models like CLIP, has been found to be vulnerable to backdoor attacks. While current backdoor defense methods primarily employ conventional data augmentation to create augmented samples aimed at feature alignment, these methods fail to capture the distinct features of backdoor samples, resulting in suboptimal defense performance. Observations reveal that adversarial examples and backdoor samples exhibit similarities in the feature space within the compromised models. Building on this insight, we propose Adversarial Backdoor Defense (ABD), a novel data augmentation strategy that aligns features with meticulously crafted adversarial examples. This approach effectively disrupts the backdoor association. Our experiments demonstrate that ABD provides robust defense against both traditional uni-modal and multimodal backdoor attacks targeting CLIP. Compared to the current state-of-the-art defense method, CleanCLIP, ABD reduces the attack success rate by 8.66% for BadNet, 10.52% for Blended, and 53.64% for BadCLIP, while maintaining a minimal average decrease of just 1.73% in clean accuracy.