Improved Membership Inference Attacks Against Language Classification Models

TL;DR

This paper introduces an improved framework for membership inference attacks on classification models, leveraging ensemble techniques to enhance privacy risk assessment accuracy in AI systems.

Contribution

The paper proposes a novel ensemble-based framework for membership inference attacks that outperforms existing methods on language and classical classification tasks.

Findings

Ensemble attack models achieve higher accuracy than single models.

The approach improves privacy risk detection in language classification.

Framework is effective on both classical and language tasks.

Abstract

Artificial intelligence systems are prevalent in everyday life, with use cases in retail, manufacturing, health, and many other fields. With the rise in AI adoption, associated risks have been identified, including privacy risks to the people whose data was used to train models. Assessing the privacy risks of machine learning models is crucial to enabling knowledgeable decisions on whether to use, deploy, or share a model. A common approach to privacy risk assessment is to run one or more known attacks against the model and measure their success rate. We present a novel framework for running membership inference attacks against classification models. Our framework takes advantage of the ensemble method, generating many specialized attack models for different subsets of the data. We show that this approach achieves higher accuracy than either a single attack model or an attack model per class label, both on classical and language classification tasks.