Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks

TL;DR

This paper introduces an incompatibility-based clustering method to detect and remove poisoned data in training datasets, significantly reducing backdoor attack success rates with minimal impact on model accuracy.

Contribution

The paper presents a novel clustering approach based on data incompatibility during training, specifically designed to defend against backdoor poisoning attacks in neural networks.

Findings

Successfully identifies poisoned data in datasets.

Reduces attack success rate to below 1% in most scenarios.

Maintains high clean accuracy with minimal drop.

Abstract

We propose a novel clustering mechanism based on an incompatibility property between subsets of data that emerges during model training. This mechanism partitions the dataset into subsets that generalize only to themselves, i.e., training on one subset does not improve performance on the other subsets. Leveraging the interaction between the dataset and the training process, our clustering mechanism partitions datasets into clusters that are defined by--and therefore meaningful to--the objective of the training process. We apply our clustering mechanism to defend against data poisoning attacks, in which the attacker injects malicious poisoned data into the training dataset to affect the trained model's output. Our evaluation focuses on backdoor attacks against deep neural networks trained to perform image classification using the GTSRB and CIFAR-10 datasets. Our results show that (1) these attacks produce poisoned datasets in which the poisoned and clean data are incompatible and (2) our technique successfully identifies (and removes) the poisoned data. In an end-to-end evaluation, our defense reduces the attack success rate to below 1% on 134 out of 165 scenarios, with only a 2% drop in clean accuracy on CIFAR-10 and a negligible drop in clean accuracy on GTSRB.