Abstract Gradient Training: A Unified Certification Framework for Data Poisoning, Unlearning, and Differential Privacy

TL;DR

This paper introduces Abstract Gradient Training (AGT), a unified certification framework that provides formal guarantees for model robustness against training data perturbations such as poisoning, unlearning, and differential privacy.

Contribution

AGT offers a novel, unified approach to certifying model robustness against various training data perturbations through parameter-space bounds.

Findings

Provides formal certification bounds for data poisoning.

Certifies model behavior under data removal and addition.

Unifies robustness certification for multiple data perturbation scenarios.

Abstract

The impact of inference-time data perturbation (e.g., adversarial attacks) has been extensively studied in machine learning, leading to well-established certification techniques for adversarial robustness. In contrast, certifying models against training data perturbations remains a relatively under-explored area. These perturbations can arise in three critical contexts: adversarial data poisoning, where an adversary manipulates training samples to corrupt model performance; machine unlearning, which requires certifying model behavior under the removal of specific training data; and differential privacy, where guarantees must be given with respect to substituting individual data points. This work introduces Abstract Gradient Training (AGT), a unified framework for certifying robustness of a given model and training procedure to training data perturbations, including bounded perturbations, the removal of data points, and the addition of new samples. By bounding the reachable set of parameters, i.e., establishing provable parameter-space bounds, AGT provides a formal approach to analyzing the behavior of models trained via first-order optimization methods.