Concept-Aware Privacy Mechanisms for Defending Embedding Inversion Attacks

TL;DR

This paper introduces SPARSE, a concept-aware privacy framework for text embeddings that selectively protects sensitive information by identifying privacy-sensitive dimensions and applying calibrated elliptical noise, improving privacy and utility.

Contribution

SPARSE is a novel user-centric framework that combines differentiable mask learning and the Mahalanobis mechanism for concept-specific privacy protection in text embeddings.

Findings

SPARSE reduces privacy leakage across multiple datasets and models.

SPARSE outperforms existing differential privacy methods in utility.

SPARSE effectively balances privacy and utility in embedding inversion attacks.

Abstract

Text embeddings enable numerous NLP applications but face severe privacy risks from embedding inversion attacks, which can expose sensitive attributes or reconstruct raw text. Existing differential privacy defenses assume uniform sensitivity across embedding dimensions, leading to excessive noise and degraded utility. We propose SPARSE, a user-centric framework for concept-specific privacy protection in text embeddings. SPARSE combines (1) differentiable mask learning to identify privacy-sensitive dimensions for user-defined concepts, and (2) the Mahalanobis mechanism that applies elliptical noise calibrated by dimension sensitivity. Unlike traditional spherical noise injection, SPARSE selectively perturbs privacy-sensitive dimensions while preserving non-sensitive semantics. Evaluated across six datasets with three embedding models and attack scenarios, SPARSE consistently reduces privacy leakage while achieving superior downstream performance compared to state-of-the-art DP methods.