The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts

TL;DR

This paper presents ProSan, an end-to-end framework that anonymizes prompts in online LLM applications, balancing privacy protection with task usability and adaptability across different computational resources.

Contribution

ProSan is a novel prompt sanitization framework that dynamically adjusts privacy levels, maintains readability, and integrates seamlessly into online LLM services.

Findings

Effectively removes private information in various tasks

Maintains high task performance with minimal accuracy loss

Adapts to diverse computational resource conditions

Abstract

The rapid adoption of online chatbots represents a significant advancement in artificial intelligence. However, this convenience brings considerable privacy concerns, as prompts can inadvertently contain sensitive information exposed to large language models (LLMs). Limited by high computational costs, reduced task usability, and excessive system modifications, previous works based on local deployment, embedding perturbation, and homomorphic encryption are inapplicable to online prompt-based LLM applications. To address these issues, this paper introduces Prompt Privacy Sanitizer (i.e., ProSan), an end-to-end prompt privacy protection framework that can produce anonymized prompts with contextual privacy removed while maintaining task usability and human readability. It can also be seamlessly integrated into the online LLM service pipeline. To achieve high usability and dynamic anonymity, ProSan flexibly adjusts its protection targets and strength based on the importance of the words and the privacy leakage risk of the prompts. Additionally, ProSan is capable of adapting to diverse computational resource conditions, ensuring privacy protection even for mobile devices with limited computing power. Our experiments demonstrate that ProSan effectively removes private information across various tasks, including question answering, text summarization, and code generation, with minimal reduction in task performance.