Mutual Information Optimally Local Private Discrete Distribution Estimation

TL;DR

This paper investigates the optimal mutual information bounds for local differential privacy in discrete distribution estimation, introducing a $k$-subset mechanism that maximizes data utility while preserving privacy.

Contribution

It provides the exact mutual information bound and proposes an efficient, optimal $k$-subset mechanism for local privacy-preserving discrete distribution estimation.

Findings

Derived the exact mutual information bound under local $$-differential privacy.

Proposed an efficient implementation of the $k$-subset mechanism.

Showed the mechanism's optimality over existing approaches.

Abstract

Consider statistical learning (e.g. discrete distribution estimation) with local $\epsilon$-differential privacy, which preserves each data provider's privacy locally, we aim to optimize statistical data utility under the privacy constraints. Specifically, we study maximizing mutual information between a provider's data and its private view, and give the exact mutual information bound along with an attainable mechanism: $k$-subset mechanism as results. The mutual information optimal mechanism randomly outputs a size $k$ subset of the original data domain with delicate probability assignment, where $k$ varies with the privacy level $\epsilon$ and the data domain size $d$. After analysing the limitations of existing local private mechanisms from mutual information perspective, we propose an efficient implementation of the $k$-subset mechanism for discrete distribution estimation, and show its optimality guarantees over existing approaches.