Whistledown: Combining User-Level Privacy with Conversational Coherence in LLMs

TL;DR

Whistledown is a privacy layer for LLMs that protects user PII through pseudonymization and local differential privacy, enabling private, coherent conversations without API modifications.

Contribution

It introduces Whistledown, a novel privacy-preserving method combining pseudonymization and $$-LDP with caching, deployable on user devices or enterprise gateways.

Findings

Provides strong privacy guarantees with low overhead.

Maintains conversational coherence despite privacy transformations.

Compatible with existing LLM APIs.

Abstract

Users increasingly rely on large language models (LLMs) for personal, emotionally charged, and socially sensitive conversations. However, prompts sent to cloud-hosted models can contain personally identifiable information (PII) that users do not want logged, retained, or leaked. We observe this to be especially acute when users discuss friends, coworkers, or adversaries, i.e., when they spill the tea. Enterprises face the same challenge when they want to use LLMs for internal communication and decision-making. In this whitepaper, we present Whistledown, a best-effort privacy layer that modifies prompts before they are sent to the LLM. Whistledown combines pseudonymization and $\epsilon$-local differential privacy ($\epsilon$-LDP) with transformation caching to provide best-effort privacy protection without sacrificing conversational utility. Whistledown is designed to have low compute and memory overhead, allowing it to be deployed directly on a client's device in the case of individual users. For enterprise users, Whistledown is deployed centrally within a zero-trust gateway that runs on an enterprise's trusted infrastructure. Whistledown requires no changes to the existing APIs of popular LLM providers.