Differentially Private Distributed Bayesian Linear Regression with MCMC

TL;DR

This paper introduces a new Bayesian inference framework for distributed linear regression that ensures differential privacy using noisy summary statistics and employs MCMC algorithms for efficient Bayesian estimation.

Contribution

It develops a novel generative model for private shared statistics and offers both iterative and one-step Bayesian estimation methods with computational advantages.

Findings

Effective privacy-preserving Bayesian linear regression demonstrated on real and simulated data

Proposed methods achieve accurate estimation and prediction

Fast Bayesian estimation method reduces computational load

Abstract

We propose a novel Bayesian inference framework for distributed differentially private linear regression. We consider a distributed setting where multiple parties hold parts of the data and share certain summary statistics of their portions in privacy-preserving noise. We develop a novel generative statistical model for privately shared statistics, which exploits a useful distributional relation between the summary statistics of linear regression. Bayesian estimation of the regression coefficients is conducted mainly using Markov chain Monte Carlo algorithms, while we also provide a fast version to perform Bayesian estimation in one iteration. The proposed methods have computational advantages over their competitors. We provide numerical results on both real and simulated data, which demonstrate that the proposed algorithms provide well-rounded estimation and prediction.