Membership Inference Attacks Against In-Context Learning

TL;DR

This paper introduces the first membership inference attack against In-Context Learning in large language models, demonstrating high accuracy and proposing defenses to mitigate privacy risks.

Contribution

It presents novel attack strategies for ICL, evaluates their effectiveness across models, and explores defense mechanisms to enhance privacy protection.

Findings

Attacks achieve up to 95% accuracy in membership inference.

Hybrid attack outperforms individual strategies in most cases.

Combining multiple defenses significantly reduces privacy leakage.

Abstract

Adapting Large Language Models (LLMs) to specific tasks introduces concerns about computational efficiency, prompting an exploration of efficient methods such as In-Context Learning (ICL). However, the vulnerability of ICL to privacy attacks under realistic assumptions remains largely unexplored. In this work, we present the first membership inference attack tailored for ICL, relying solely on generated texts without their associated probabilities. We propose four attack strategies tailored to various constrained scenarios and conduct extensive experiments on four popular large language models. Empirical results show that our attacks can accurately determine membership status in most cases, e.g., 95\% accuracy advantage against LLaMA, indicating that the associated risks are much higher than those shown by existing probability-based attacks. Additionally, we propose a hybrid attack that synthesizes the strengths of the aforementioned strategies, achieving an accuracy advantage of over 95\% in most cases. Furthermore, we investigate three potential defenses targeting data, instruction, and output. Results demonstrate combining defenses from orthogonal dimensions significantly reduces privacy leakage and offers enhanced privacy assurances.