Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models

TL;DR

This paper demonstrates that membership inference attacks can succeed on large language models when testing involves multiple documents, introducing new benchmarks and methods that enable such attacks at larger data scales.

Contribution

It introduces new benchmarks and adaptation of dataset inference techniques to successfully perform membership inference on large language models at document and collection levels.

Findings

MIA works on LLMs with multiple documents.

New benchmarks measure MIA performance across data scales.

Adapted methods enable successful MIA on pre-trained and fine-tuned LLMs.

Abstract

Membership inference attacks (MIA) attempt to verify the membership of a given data sample in the training set for a model. MIA has become relevant in recent years, following the rapid development of large language models (LLM). Many are concerned about the usage of copyrighted materials for training them and call for methods for detecting such usage. However, recent research has largely concluded that current MIA methods do not work on LLMs. Even when they seem to work, it is usually because of the ill-designed experimental setup where other shortcut features enable "cheating." In this work, we argue that MIA still works on LLMs, but only when multiple documents are presented for testing. We construct new benchmarks that measure the MIA performances at a continuous scale of data samples, from sentences (n-grams) to a collection of documents (multiple chunks of tokens). To validate the efficacy of current MIA approaches at greater scales, we adapt a recent work on Dataset Inference (DI) for the task of binary membership detection that aggregates paragraph-level MIA features to enable MIA at document and collection of documents level. This baseline achieves the first successful MIA on pre-trained and fine-tuned LLMs.