Knowledge Cross-Distillation for Membership Privacy

TL;DR

This paper introduces a novel knowledge distillation-based defense against membership inference attacks that does not require public data, maintaining high privacy protection and model accuracy in sensitive domains.

Contribution

The paper proposes a new membership privacy defense using knowledge distillation without relying on public data, addressing limitations of existing methods.

Findings

Comparable privacy protection and accuracy to DMP on tabular datasets

Better privacy-utility trade-off than existing public-data-free defenses on CIFAR10

Effective against membership inference attacks in privacy-sensitive applications

Abstract

A membership inference attack (MIA) poses privacy risks for the training data of a machine learning model. With an MIA, an attacker guesses if the target data are a member of the training dataset. The state-of-the-art defense against MIAs, distillation for membership privacy (DMP), requires not only private data for protection but a large amount of unlabeled public data. However, in certain privacy-sensitive domains, such as medicine and finance, the availability of public data is not guaranteed. Moreover, a trivial method for generating public data by using generative adversarial networks significantly decreases the model accuracy, as reported by the authors of DMP. To overcome this problem, we propose a novel defense against MIAs that uses knowledge distillation without requiring public data. Our experiments show that the privacy protection and accuracy of our defense are comparable to those of DMP for the benchmark tabular datasets used in MIA research, Purchase100 and Texas100, and our defense has a much better privacy-utility trade-off than those of the existing defenses that also do not use public data for the image dataset CIFAR10.