Privacy-Preserving Distributed Processing: Metrics, Bounds, and Algorithms

TL;DR

This paper introduces information-theoretic metrics to compare privacy-preserving distributed algorithms, derives bounds on privacy, and evaluates various methods through theoretical and numerical analysis to guide algorithm design.

Contribution

It proposes new metrics for analyzing privacy algorithms, establishes a lower bound on privacy, and compares existing methods both theoretically and numerically.

Findings

Metrics enable comparison of privacy algorithms

Lower bound provides insights into privacy limits

Numerical validation compares state-of-the-art approaches

Abstract

Privacy-preserving distributed processing has recently attracted considerable attention. It aims to design solutions for conducting signal processing tasks over networks in a decentralized fashion without violating privacy. Many algorithms can be adopted to solve this problem such as differential privacy, secure multiparty computation, and the recently proposed distributed optimization based subspace perturbation. However, how these algorithms relate to each other is not fully explored yet. In this paper, we therefore first propose information-theoretic metrics based on mutual information. Using the proposed metrics, we are able to compare and relate a number of existing well-known algorithms. We then derive a lower bound on individual privacy that gives insights on the nature of the problem. To validate the above claims, we investigate a concrete example and compare a number of state-of-the-art approaches in terms of different aspects such as output utility, individual privacy and algorithm robustness against the number of corrupted parties, using not only theoretical analysis but also numerical validation. Finally, we discuss and provide principles for designing appropriate algorithms for different applications.