Shortcuts Arising from Contrast: Effective and Covert Clean-Label Attacks in Prompt-Based Learning

TL;DR

This paper introduces Contrastive Shortcut Injection (CSI), a novel clean-label attack method that exploits contrast to create stealthy backdoor triggers in prompt-based learning, demonstrating high effectiveness and stealthiness in text classification.

Contribution

The paper proposes CSI, a new contrastive approach for clean-label backdoor attacks in prompt-based learning, improving stealthiness and effectiveness at low poisoning rates.

Findings

CSI achieves high attack success rates in text classification.

CSI maintains high stealthiness with low poisoning rates.

Effectiveness varies between full-shot and few-shot settings.

Abstract

Prompt-based learning paradigm has demonstrated remarkable efficacy in enhancing the adaptability of pretrained language models (PLMs), particularly in few-shot scenarios. However, this learning paradigm has been shown to be vulnerable to backdoor attacks. The current clean-label attack, employing a specific prompt as a trigger, can achieve success without the need for external triggers and ensure correct labeling of poisoned samples, which is more stealthy compared to the poisoned-label attack, but on the other hand, it faces significant issues with false activations and poses greater challenges, necessitating a higher rate of poisoning. Using conventional negative data augmentation methods, we discovered that it is challenging to trade off between effectiveness and stealthiness in a clean-label setting. In addressing this issue, we are inspired by the notion that a backdoor acts as a shortcut and posit that this shortcut stems from the contrast between the trigger and the data utilized for poisoning. In this study, we propose a method named Contrastive Shortcut Injection (CSI), by leveraging activation values, integrates trigger design and data selection strategies to craft stronger shortcut features. With extensive experiments on full-shot and few-shot text classification tasks, we empirically validate CSI's high effectiveness and high stealthiness at low poisoning rates. Notably, we found that the two approaches play leading roles in full-shot and few-shot settings, respectively.