Towards Differentially Private Text Representations

TL;DR

This paper introduces a new deep learning framework for text representations that preserves privacy under untrusted servers using a novel local differential privacy protocol, achieving high accuracy.

Contribution

It presents a new deep learning framework with a novel local differential privacy protocol that reduces accuracy impact and offers greater flexibility, outperforming existing methods.

Findings

Comparable or better performance than non-private frameworks

Enhanced privacy with minimal accuracy loss

Flexible randomization probabilities improve privacy-utility trade-off

Abstract

Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter $\epsilon$ on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.