Easy Data Unlearning Bench

TL;DR

This paper introduces a standardized, easy-to-use benchmarking suite for evaluating machine unlearning methods, facilitating reproducibility, scalability, and fair comparison across different algorithms.

Contribution

We present a unified benchmarking framework with precomputed models and standardized metrics to simplify and accelerate research in machine unlearning.

Findings

Provides a comprehensive, extensible benchmark suite

Enables reproducible and fair comparison of unlearning methods

Facilitates faster development and evaluation of unlearning algorithms

Abstract

Evaluating machine unlearning methods remains technically challenging, with recent benchmarks requiring complex setups and significant engineering overhead. We introduce a unified and extensible benchmarking suite that simplifies the evaluation of unlearning algorithms using the KLoM (KL divergence of Margins) metric. Our framework provides precomputed model ensembles, oracle outputs, and streamlined infrastructure for running evaluations out of the box. By standardizing setup and metrics, it enables reproducible, scalable, and fair comparison across unlearning methods. We aim for this benchmark to serve as a practical foundation for accelerating research and promoting best practices in machine unlearning. Our code and data are publicly available.