Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text

TL;DR

This paper introduces a method using hyperbolic space representations to enhance text privacy while maintaining utility, providing strong privacy guarantees and minimal impact on downstream tasks.

Contribution

It proposes a novel hyperbolic space perturbation technique satisfying dx-privacy, improving privacy guarantees over Euclidean methods in text data.

Findings

Over 20x greater privacy guarantees compared to Euclidean baseline.

Effective protection against authorship attribution algorithms.

Minimal utility loss on downstream machine learning models.

Abstract

Guaranteeing a certain level of user privacy in an arbitrary piece of text is a challenging issue. However, with this challenge comes the potential of unlocking access to vast data stores for training machine learning models and supporting data driven decisions. We address this problem through the lens of dx-privacy, a generalization of Differential Privacy to non Hamming distance metrics. In this work, we explore word representations in Hyperbolic space as a means of preserving privacy in text. We provide a proof satisfying dx-privacy, then we define a probability distribution in Hyperbolic space and describe a way to sample from it in high dimensions. Privacy is provided by perturbing vector representations of words in high dimensional Hyperbolic space to obtain a semantic generalization. We conduct a series of experiments to demonstrate the tradeoff between privacy and utility. Our privacy experiments illustrate protections against an authorship attribution algorithm while our utility experiments highlight the minimal impact of our perturbations on several downstream machine learning models. Compared to the Euclidean baseline, we observe > 20x greater guarantees on expected privacy against comparable worst case statistics.