Backdooring Self-Supervised Contrastive Learning by Noisy Alignment

TL;DR

This paper introduces Noisy Alignment, a novel data poisoning attack on self-supervised contrastive learning that suppresses noise in poisoned images to achieve state-of-the-art backdoor effectiveness while resisting defenses.

Contribution

It proposes a new DPCL method that manipulates contrastive learning's cropping process, improving attack success and robustness over existing methods.

Findings

Achieves state-of-the-art backdoor attack success rates.

Maintains high clean-data accuracy.

Resists common backdoor defenses.

Abstract

Self-supervised contrastive learning (CL) effectively learns transferable representations from unlabeled data containing images or image-text pairs but suffers vulnerability to data poisoning backdoor attacks (DPCLs). An adversary can inject poisoned images into pretraining datasets, causing compromised CL encoders to exhibit targeted misbehavior in downstream tasks. Existing DPCLs, however, achieve limited efficacy due to their dependence on fragile implicit co-occurrence between backdoor and target object and inadequate suppression of discriminative features in backdoored images. We propose Noisy Alignment (NA), a DPCL method that explicitly suppresses noise components in poisoned images. Inspired by powerful training-controllable CL attacks, we identify and extract the critical objective of noisy alignment, adapting it effectively into data-poisoning scenarios. Our method implements noisy alignment by strategically manipulating contrastive learning's random cropping mechanism, formulating this process as an image layout optimization problem with theoretically derived optimal parameters. The resulting method is simple yet effective, achieving state-of-the-art performance compared to existing DPCLs, while maintaining clean-data accuracy. Furthermore, Noisy Alignment demonstrates robustness against common backdoor defenses. Codes can be found at https://github.com/jsrdcht/Noisy-Alignment.