Turning Black Box into White Box: Dataset Distillation Leaks

TL;DR

This paper reveals that dataset distillation methods, while compressing data effectively, can leak sensitive information through the synthetic data, exposing privacy risks and enabling model and data inference attacks.

Contribution

The paper introduces the Information Revelation Attack (IRA), demonstrating privacy vulnerabilities in existing dataset distillation techniques and exposing the implicit encoding of model trajectories.

Findings

IRA can accurately identify distillation algorithms and model architectures.

IRA can infer membership of data samples in the original dataset.

IRA can recover sensitive data samples from the real dataset.

Abstract

Dataset distillation compresses a large real dataset into a small synthetic one, enabling models trained on the synthetic data to achieve performance comparable to those trained on the real data. Although synthetic datasets are assumed to be privacy-preserving, we show that existing distillation methods can cause severe privacy leakage because synthetic datasets implicitly encode the weight trajectories of the distilled model, they become over-informative and exploitable by adversaries. To expose this risk, we introduce the Information Revelation Attack (IRA) against state-of-the-art distillation techniques. Experiments show that IRA accurately predicts both the distillation algorithm and model architecture, and can successfully infer membership and recover sensitive samples from the real dataset.