SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks

TL;DR

This paper evaluates the privacy risks of fine-tuned large language models against membership inference attacks and introduces SOFT, a selective data obfuscation method that reduces privacy leakage while preserving model utility.

Contribution

The paper provides the first comprehensive analysis of MIA vulnerabilities in fine-tuned LLMs and proposes SOFT, a novel selective obfuscation technique to enhance privacy protection.

Findings

MIAs exploit loss reduction during fine-tuning effectively.

SOFT reduces privacy risks across multiple domains and models.

Model utility is maintained with the proposed defense.

Abstract

Large language models (LLMs) have achieved remarkable success and are widely adopted for diverse applications. However, fine-tuning these models often involves private or sensitive information, raising critical privacy concerns. In this work, we conduct the first comprehensive study evaluating the vulnerability of fine-tuned LLMs to membership inference attacks (MIAs). Our empirical analysis demonstrates that MIAs exploit the loss reduction during fine-tuning, making them highly effective in revealing membership information. These findings motivate the development of our defense. We propose SOFT (\textbf{S}elective data \textbf{O}bfuscation in LLM \textbf{F}ine-\textbf{T}uning), a novel defense technique that mitigates privacy leakage by leveraging influential data selection with an adjustable parameter to balance utility preservation and privacy protection. Our extensive experiments span six diverse domains and multiple LLM architectures and scales. Results show that SOFT effectively reduces privacy risks while maintaining competitive model performance, offering a practical and scalable solution to safeguard sensitive information in fine-tuned LLMs.