Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactions

TL;DR

This paper introduces a middleware system that uses a small language model to predict the utility of sanitized prompts, aiming to reduce resource waste and improve privacy in large language model interactions.

Contribution

It proposes a novel middleware architecture leveraging a small language model to predict prompt utility before querying the main LLM, addressing privacy and resource efficiency.

Findings

Prevents resource waste for up to 20% of prompts.

Shows the impact of implementation choices on sanitization performance.

Reproduces and critiques existing text sanitization experiments.

Abstract

Interactions with online Large Language Models raise privacy issues where providers can gather sensitive information about users and their companies from the prompts. While textual prompts can be sanitized using Differential Privacy, we show that it is difficult to anticipate the performance of an LLM on such sanitized prompt. Poor performance has clear monetary consequences for LLM services charging on a pay-per-use model as well as great amount of computing resources wasted. To this end, we propose a middleware architecture leveraging a Small Language Model to predict the utility of a given sanitized prompt before it is sent to the LLM. We experimented on a summarization task and a translation task to show that our architecture helps prevent such resource waste for up to 20% of the prompts. During our study, we also reproduced experiments from one of the most cited paper on text sanitization using DP and show that a potential performance-driven implementation choice dramatically changes the output while not being explicitly acknowledged in the paper.