Data Lineage Inference: Uncovering Privacy Vulnerabilities of Dataset Pruning

TL;DR

This paper uncovers privacy vulnerabilities in dataset pruning for machine learning, demonstrating that redundant data can be inferred even before model training, and introduces a new data-centric inference paradigm with practical attack methods.

Contribution

It introduces Data Lineage Inference (DaLI), a novel paradigm for privacy inference in dataset pruning, along with four new attack methods and a metric for privacy risk assessment.

Findings

Redundant data can be identified before model training.

Different pruning methods have varying privacy risks.

A new metric, Brimming score, guides privacy-preserving pruning.

Abstract

In this work, we systematically explore the data privacy issues of dataset pruning in machine learning systems. Our findings reveal, for the first time, that even if data in the redundant set is solely used before model training, its pruning-phase membership status can still be detected through attacks. Since this is a fully upstream process before model training, traditional model output-based privacy inference methods are completely unsuitable. To address this, we introduce a new task called Data-Centric Membership Inference and propose the first ever data-centric privacy inference paradigm named Data Lineage Inference (DaLI). Under this paradigm, four threshold-based attacks are proposed, named WhoDis, CumDis, ArraDis and SpiDis. We show that even without access to downstream models, adversaries can accurately identify the redundant set with only limited prior knowledge. Furthermore, we find that different pruning methods involve varying levels of privacy leakage, and even the same pruning method can present different privacy risks at different pruning fractions. We conducted an in-depth analysis of these phenomena and introduced a metric called the Brimming score to offer guidance for selecting pruning methods with privacy protection in mind.