Differentially Private Bayesian Linear Regression

TL;DR

This paper develops noise-aware Bayesian linear regression methods that accurately produce posterior distributions under differential privacy, addressing limitations of naive approaches and enabling uncertainty quantification with sensitive data.

Contribution

It introduces novel noise-aware inference techniques for Bayesian linear regression that correctly account for privacy-induced noise, improving posterior accuracy.

Findings

Naive methods fail to produce accurate posteriors with privacy noise.

Proposed methods yield correct uncertainty quantification across various scenarios.

Approach enhances privacy-preserving data analysis in sensitive applications.

Abstract

Linear regression is an important tool across many fields that work with sensitive human-sourced data. Significant prior work has focused on producing differentially private point estimates, which provide a privacy guarantee to individuals while still allowing modelers to draw insights from data by estimating regression coefficients. We investigate the problem of Bayesian linear regression, with the goal of computing posterior distributions that correctly quantify uncertainty given privately released statistics. We show that a naive approach that ignores the noise injected by the privacy mechanism does a poor job in realistic data settings. We then develop noise-aware methods that perform inference over the privacy mechanism and produce correct posteriors across a wide range of scenarios.