An Information Theoretic approach to Post Randomization Methods under Differential Privacy

TL;DR

This paper introduces an information-theoretic method for selecting post-randomization matrices in data privacy, balancing data utility and privacy guarantees under differential privacy constraints.

Contribution

It formulates the selection of the randomization matrix as a constrained maximization of mutual information, solvable via convex linear programming, enhancing privacy-utility trade-offs.

Findings

Maximizes mutual information under differential privacy constraints.

Reduces the problem to convex linear programming.

Provides a practical optimization framework for PRAM.

Abstract

Post Randomization Methods (PRAM) are among the most popular disclosure limitation techniques for both categorical and continuous data. In the categorical case, given a stochastic matrix $M$ and a specified variable, an individual belonging to category $i$ is changed to category $j$ with probability $M_{i,j}$. Every approach to choose the randomization matrix $M$ has to balance between two desiderata: 1) preserving as much statistical information from the raw data as possible; 2) guaranteeing the privacy of individuals in the dataset. This trade-off has generally been shown to be very challenging to solve. In this work, we use recent tools from the computer science literature and propose to choose $M$ as the solution of a constrained maximization problems. Specifically, $M$ is chosen as the solution of a constrained maximization problem, where we maximize the Mutual Information between raw and transformed data, given the constraint that the transformation satisfies the notion of Differential Privacy. For the general Categorical model, it is shown how this maximization problem reduces to a convex linear programming and can be therefore solved with known optimization algorithms.