Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning

TL;DR

This paper reveals that data curation processes in machine learning can leak private information through various stages, even when models are trained only on curated public data, and proposes privacy-preserving adaptations.

Contribution

It introduces novel membership inference attacks targeting curation steps and demonstrates that privacy risks exist beyond model training, proposing differentially private curation methods as mitigation.

Findings

Curation stages leak private data information.

Models trained on curated data can still reveal private membership.

Differential privacy effectively reduces leakage.

Abstract

In machine learning, curation is used to select the most valuable data for improving both model accuracy and computational efficiency. Recently, curation has also been explored as a solution for private machine learning: rather than training directly on sensitive data, which is known to leak information through model predictions, the private data is used only to guide the selection of useful public data. The resulting model is then trained solely on curated public data. It is tempting to assume that such a model is privacy-preserving because it has never seen the private data. Yet, we show that without further protection, curation pipelines can still leak private information. Specifically, we introduce novel attacks against popular curation methods, targeting every major step: the computation of curation scores, the selection of the curated subset, and the final trained model. We demonstrate that each stage reveals information about the private dataset and that even models trained exclusively on curated public data leak membership information about the private data that guided curation. These findings highlight the previously overlooked inherent privacy risks of data curation and show that privacy assessment must extend beyond the training procedure to include the data selection process. Our differentially private adaptations of curation methods effectively mitigate leakage, indicating that formal privacy guarantees for curation are a promising direction.