Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning

TL;DR

This paper introduces a new framework for certifying the robustness of learning algorithms against adaptive, online data poisoning attacks, addressing a gap in existing static adversary models, with initial applications to mean estimation and binary classification.

Contribution

It proposes a novel method to compute certified bounds on the impact of dynamic poisoning attacks and designs robust learning algorithms based on these certificates.

Findings

Framework for certifying robustness against adaptive attacks

Application to mean estimation and binary classification

Open directions for extending the approach

Abstract

The rise of foundation models fine-tuned on human feedback from potentially untrusted users has increased the risk of adversarial data poisoning, necessitating the study of robustness of learning algorithms against such attacks. Existing research on provable certified robustness against data poisoning attacks primarily focuses on certifying robustness for static adversaries who modify a fraction of the dataset used to train the model before the training algorithm is applied. In practice, particularly when learning from human feedback in an online sense, adversaries can observe and react to the learning process and inject poisoned samples that optimize adversarial objectives better than when they are restricted to poisoning a static dataset once, before the learning algorithm is applied. Indeed, it has been shown in prior work that online dynamic adversaries can be significantly more powerful than static ones. We present a novel framework for computing certified bounds on the impact of dynamic poisoning, and use these certificates to design robust learning algorithms. We give an illustration of the framework for the mean estimation and binary classification problems and outline directions for extending this in further work. The code to implement our certificates and replicate our results is available at https://github.com/Avinandan22/Certified-Robustness.