Membership Inference Attacks on Sequence Models

TL;DR

This paper adapts membership inference attacks to better assess privacy risks in sequence models by modeling within-sequence correlations, improving audit effectiveness without extra computational costs.

Contribution

It introduces a method to extend existing membership inference attacks to sequence models by explicitly modeling sequential correlations, enhancing privacy leakage detection.

Findings

Improved effectiveness of memorization audits for sequence models

No additional computational costs introduced

Demonstrated through a case study

Abstract

Sequence models, such as Large Language Models (LLMs) and autoregressive image generators, have a tendency to memorize and inadvertently leak sensitive information. While this tendency has critical legal implications, existing tools are insufficient to audit the resulting risks. We hypothesize that those tools' shortcomings are due to mismatched assumptions. Thus, we argue that effectively measuring privacy leakage in sequence models requires leveraging the correlations inherent in sequential generation. To illustrate this, we adapt a state-of-the-art membership inference attack to explicitly model within-sequence correlations, thereby demonstrating how a strong existing attack can be naturally extended to suit the structure of sequence models. Through a case study, we show that our adaptations consistently improve the effectiveness of memorization audits without introducing additional computational costs. Our work hence serves as an important stepping stone toward reliable memorization audits for large sequence models.