Gaussian Processes with Differential Privacy

TL;DR

This paper introduces a method for applying differential privacy to Gaussian processes, protecting both inputs and outputs, and demonstrates its effectiveness with accurate models under strong privacy constraints.

Contribution

The paper presents a novel approach to achieve differential privacy for both inputs and outputs of Gaussian processes using sparse GP techniques and private hyperparameter learning.

Findings

Effective privacy protection with accurate predictions

Method works well with sufficient data

Maintains model utility under strong privacy constraints

Abstract

Gaussian processes (GPs) are non-parametric Bayesian models that are widely used for diverse prediction tasks. Previous work in adding strong privacy protection to GPs via differential privacy (DP) has been limited to protecting only the privacy of the prediction targets (model outputs) but not inputs. We break this limitation by introducing GPs with DP protection for both model inputs and outputs. We achieve this by using sparse GP methodology and publishing a private variational approximation on known inducing points. The approximation covariance is adjusted to approximately account for the added uncertainty from DP noise. The approximation can be used to compute arbitrary predictions using standard sparse GP techniques. We propose a method for hyperparameter learning using a private selection protocol applied to validation set log-likelihood. Our experiments demonstrate that given sufficient amount of data, the method can produce accurate models under strong privacy protection.