Learning Obfuscations Of LLM Embedding Sequences: Stained Glass Transform

TL;DR

This paper introduces the Stained Glass Transform, a learned stochastic method for transforming LLM embeddings to enhance privacy while maintaining model utility, supported by theoretical analysis and empirical validation.

Contribution

The paper proposes a novel learned stochastic embedding transformation that provides privacy guarantees for LLM inputs without sacrificing performance.

Findings

Theoretical connection to mutual information in Gaussian Mixture Models.

Empirical results show preserved LLM utility after transformation.

Privacy estimates indicate effective information hiding.

Abstract

The high cost of ownership of AI compute infrastructure and challenges of robust serving of large language models (LLMs) has led to a surge in managed Model-as-a-service deployments. Even when enterprises choose on-premises deployments, the compute infrastructure is typically shared across many teams in order to maximize the return on investment. In both scenarios the deployed models operate only on plaintext data, and so enterprise data owners must allow their data to appear in plaintext on a shared or multi-tenant compute infrastructure. This results in data owners with private or sensitive data being hesitant or restricted in what data they use with these types of deployments. In this work we introduce the Stained Glass Transform, a learned, stochastic, and sequence dependent transformation of the word embeddings of an LLM which information theoretically provides privacy to the input of the LLM while preserving the utility of model. We theoretically connect a particular class of Stained Glass Transforms to the theory of mutual information of Gaussian Mixture Models. We then calculate a-postiori privacy estimates, based on mutual information, and verify the privacy and utility of instances of transformed embeddings through token level metrics of privacy and standard LLM performance benchmarks.