PSY: Posterior Sampling Based Privacy Enhancer in Large Language Models

TL;DR

This paper introduces PSY, a privacy enhancement method for large language models using posterior sampling within LoRA, effectively reducing privacy leakage without harming model performance.

Contribution

The paper proposes a novel privacy enhancement technique called PSY that integrates posterior sampling into LoRA for LLMs, improving privacy protection.

Findings

PSY reduces attack success rates in membership inference and data extraction.

PSY maintains model performance with minimal negative impact.

Effective privacy enhancement demonstrated across multiple LLM architectures and datasets.

Abstract

Privacy vulnerabilities in LLMs, such as leakage from memorization, have been constantly identified, and various mitigation proposals have been proposed. LoRA is usually used in fine-tuning LLMs and a good entry point to insert privacy-enhancing modules. In this ongoing research, we introduce PSY, a Posterior Sampling based PrivacY enhancer that can be used in LoRA. We propose a simple yet effective realization of PSY using posterior sampling, which effectively prevents privacy leakage from intermediate information and, in turn, preserves the privacy of data owners. We evaluate LoRA extended with PSY against state-of-the-art membership inference and data extraction attacks. The experiments are executed on three different LLM architectures fine-tuned on three datasets with LoRA. In contrast to the commonly used differential privacy method, we find that our proposed modification consistently reduces the attack success rate. Meanwhile, our method has almost no negative impact on model fine-tuning or final performance. Most importantly, PSY reveals a promising path toward privacy enhancement with latent space extensions.