Detection and Defense of Unlearnable Examples

TL;DR

This paper investigates the detectability of unlearnable examples, demonstrating their easy detection with simple methods, and proposes a novel defense using data augmentation and adversarial noises to enhance robustness against such examples.

Contribution

It reveals the detectability of unlearnable examples and introduces a new defense strategy combining data augmentation and adversarial noises to improve robustness.

Findings

Unlearnable examples are linearly separable and easily detectable.

Simple network detection methods can identify all existing unlearnable examples.

The proposed defense reduces detectability and improves robustness against unlearnable examples.

Abstract

Privacy preserving has become increasingly critical with the emergence of social media. Unlearnable examples have been proposed to avoid leaking personal information on the Internet by degrading generalization abilities of deep learning models. However, our study reveals that unlearnable examples are easily detectable. We provide theoretical results on linear separability of certain unlearnable poisoned dataset and simple network based detection methods that can identify all existing unlearnable examples, as demonstrated by extensive experiments. Detectability of unlearnable examples with simple networks motivates us to design a novel defense method. We propose using stronger data augmentations coupled with adversarial noises generated by simple networks, to degrade the detectability and thus provide effective defense against unlearnable examples with a lower cost. Adversarial training with large budgets is a widely-used defense method on unlearnable examples. We establish quantitative criteria between the poison and adversarial budgets which determine the existence of robust unlearnable examples or the failure of the adversarial defense.