APBench: A Unified Benchmark for Availability Poisoning Attacks and Defenses

TL;DR

APBench is a comprehensive benchmark that evaluates the effectiveness of availability poisoning attacks and defenses across multiple datasets, models, and attack types, highlighting current limitations and providing a standardized evaluation framework.

Contribution

This paper introduces APBench, a unified benchmark for systematically assessing availability poisoning attacks and defenses, addressing the lack of standardized evaluation in this field.

Findings

Existing attacks are inadequate in protecting individual privacy.

APBench provides a standardized platform for fair comparison of methods.

Evaluation across multiple datasets and models reveals gaps in current defenses.

Abstract

The efficacy of availability poisoning, a method of poisoning data by injecting imperceptible perturbations to prevent its use in model training, has been a hot subject of investigation. Previous research suggested that it was difficult to effectively counteract such poisoning attacks. However, the introduction of various defense methods has challenged this notion. Due to the rapid progress in this field, the performance of different novel methods cannot be accurately validated due to variations in experimental setups. To further evaluate the attack and defense capabilities of these poisoning methods, we have developed a benchmark -- APBench for assessing the efficacy of adversarial poisoning. APBench consists of 9 state-of-the-art availability poisoning attacks, 8 defense algorithms, and 4 conventional data augmentation techniques. We also have set up experiments with varying different poisoning ratios, and evaluated the attacks on multiple datasets and their transferability across model architectures. We further conducted a comprehensive evaluation of 2 additional attacks specifically targeting unsupervised models. Our results reveal the glaring inadequacy of existing attacks in safeguarding individual privacy. APBench is open source and available to the deep learning community: https://github.com/lafeat/apbench.