Differentially Private Bayesian Learning on Distributed Data

TL;DR

This paper introduces a privacy-preserving Bayesian learning method for distributed data using differential privacy, secure multi-party computation, and Gaussian mechanisms, enabling effective learning without a trusted central authority.

Contribution

It presents a novel distributed DP Bayesian inference approach combining secure aggregation and Gaussian mechanisms, improving efficiency and privacy guarantees.

Findings

Achieves asymptotically optimal DP Bayesian inference

Reduces additional computational costs compared to existing methods

Enables privacy-preserving learning with minimal trust assumptions

Abstract

Many applications of machine learning, for example in health care, would benefit from methods that can guarantee privacy of data subjects. Differential privacy (DP) has become established as a standard for protecting learning results. The standard DP algorithms require a single trusted party to have access to the entire data, which is a clear weakness. We consider DP Bayesian learning in a distributed setting, where each party only holds a single sample or a few samples of the data. We propose a learning strategy based on a secure multi-party sum function for aggregating summaries from data holders and the Gaussian mechanism for DP. Our method builds on an asymptotically optimal and practically efficient DP Bayesian inference with rapidly diminishing extra cost.