Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services

TL;DR

This paper introduces LOPSIDED, a semantically-aware privacy framework that replaces sensitive personal information with pseudonyms in user prompts to protect privacy without degrading conversational quality in LLMs.

Contribution

The paper presents a novel privacy-preserving method that maintains semantic integrity by dynamically pseudonymizing entities in user prompts for conversational AI.

Findings

Reduces semantic utility errors by a factor of 5

Enhances privacy while preserving conversation quality

Effective on real-world ShareGPT conversations

Abstract

With the increasing use of conversational AI systems, there is growing concern over privacy leaks, especially when users share sensitive personal data in interactions with Large Language Models (LLMs). Conversations shared with these models may contain Personally Identifiable Information (PII), which, if exposed, could lead to security breaches or identity theft. To address this challenge, we present the Local Optimizations for Pseudonymization with Semantic Integrity Directed Entity Detection (LOPSIDED) framework, a semantically-aware privacy agent designed to safeguard sensitive PII data when using remote LLMs. Unlike prior work that often degrade response quality, our approach dynamically replaces sensitive PII entities in user prompts with semantically consistent pseudonyms, preserving the contextual integrity of conversations. Once the model generates its response, the pseudonyms are automatically depseudonymized, ensuring the user receives an accurate, privacy-preserving output. We evaluate our approach using real-world conversations sourced from ShareGPT, which we further augment and annotate to assess whether named entities are contextually relevant to the model's response. Our results show that LOPSIDED reduces semantic utility errors by a factor of 5 compared to baseline techniques, all while enhancing privacy.