TruVRF: Towards Triple-Granularity Verification on Machine Unlearning

TL;DR

TruVRF is a non-invasive framework that verifies machine unlearning at multiple granularities, ensuring honest model updates and detecting dishonest providers with high accuracy across different datasets and frameworks.

Contribution

We propose TruVRF, a novel multi-granularity verification framework for machine unlearning that detects dishonest behavior without invasive methods, applicable to various unlearning frameworks.

Findings

Over 90% accuracy for class and sample-level metrics.

Robust detection across different datasets and frameworks.

Effective in identifying neglecting, lazy, and deceiving servers.

Abstract

The concept of the right to be forgotten has led to growing interest in machine unlearning, but reliable validation methods are lacking, creating opportunities for dishonest model providers to mislead data contributors. Traditional invasive methods like backdoor injection are not feasible for legacy data. To address this, we introduce TruVRF, a non-invasive unlearning verification framework operating at class-, volume-, and sample-level granularities. TruVRF includes three Unlearning-Metrics designed to detect different types of dishonest servers: Neglecting, Lazy, and Deceiving. Unlearning-Metric-I checks class alignment, Unlearning-Metric-II verifies sample count, and Unlearning-Metric-III confirms specific sample deletion. Evaluations on three datasets show TruVRF's robust performance, with over 90% accuracy for Metrics I and III, and a 4.8% to 8.2% inference deviation for Metric II. TruVRF also demonstrates generalizability and practicality across various conditions and with state-of-the-art unlearning frameworks like SISA and Amnesiac Unlearning.