Distributed Differentially Private Mutual Information Ranking and Its Applications

TL;DR

This paper introduces DDP-MI, a distributed differentially private method for mutual information computation that enhances privacy and efficiency in large-scale data analysis tasks like feature selection and ranking.

Contribution

We propose a novel distributed differentially private MI algorithm that improves computational efficiency and privacy guarantees for large-scale datasets.

Findings

DDP-MI significantly reduces computation time compared to standard MI methods.

The approach provides strong differential privacy protections against various attacks.

Experimental results demonstrate scalability and privacy preservation on large datasets.

Abstract

Computation of Mutual Information (MI) helps understand the amount of information shared between a pair of random variables. Automated feature selection techniques based on MI ranking are regularly used to extract information from sensitive datasets exceeding petabytes in size, over millions of features and classes. Series of one-vs-all MI computations can be cascaded to produce n-fold MI results, rapidly pinpointing informative relationships. This ability to quickly pinpoint the most informative relationships from datasets of billions of users creates privacy concerns. In this paper, we present Distributed Differentially Private Mutual Information (DDP-MI), a privacy-safe fast batch MI, across various scenarios such as feature selection, segmentation, ranking, and query expansion. This distributed implementation is protected with global model differential privacy to provide strong assurances against a wide range of privacy attacks. We also show that our DDP-MI can substantially improve the efficiency of MI calculations compared to standard implementations on a large-scale public dataset.