Information Extraction Under Privacy Constraints

TL;DR

This paper investigates how to maximize information extraction from a variable Y about Y itself while preserving privacy of another correlated variable X, analyzing properties, closed-form solutions, and asymptotic behavior under different privacy measures.

Contribution

It introduces the rate-privacy function, analyzes its properties, provides closed-form expressions for certain distributions, and studies its asymptotic behavior in continuous cases.

Findings

Rate-privacy function quantifies maximum extractable information under privacy constraints.

Closed-form expression derived for a large family of joint distributions.

Asymptotic analysis reveals differences between discrete and continuous cases.

Abstract

A privacy-constrained information extraction problem is considered where for a pair of correlated discrete random variables $(X,Y)$ governed by a given joint distribution, an agent observes $Y$ and wants to convey to a potentially public user as much information about $Y$ as possible without compromising the amount of information revealed about $X$. To this end, the so-called {\em rate-privacy function} is introduced to quantify the maximal amount of information (measured in terms of mutual information) that can be extracted from $Y$ under a privacy constraint between $X$ and the extracted information, where privacy is measured using either mutual information or maximal correlation. Properties of the rate-privacy function are analyzed and information-theoretic and estimation-theoretic interpretations of it are presented for both the mutual information and maximal correlation privacy measures. It is also shown that the rate-privacy function admits a closed-form expression for a large family of joint distributions of $(X,Y)$. Finally, the rate-privacy function under the mutual information privacy measure is considered for the case where $(X,Y)$ has a joint probability density function by studying the problem where the extracted information is a uniform quantization of $Y$ corrupted by additive Gaussian noise. The asymptotic behavior of the rate-privacy function is studied as the quantization resolution grows without bound and it is observed that not all of the properties of the rate-privacy function carry over from the discrete to the continuous case.