Strongly universally consistent nonparametric regression and classification with privatised data

TL;DR

This paper develops a new nonparametric regression and classification method that remains consistent under local differential privacy constraints by adding Laplace noise to data and using a privatized partitioning estimator.

Contribution

It introduces a novel privatized regression estimator and classification rule that are strongly universally consistent under local differential privacy constraints.

Findings

Estimator is strongly universally consistent.

Applicable to binary classification with privacy guarantees.

Uses Laplace noise on discretized features and responses.

Abstract

In this paper we revisit the classical problem of nonparametric regression, but impose local differential privacy constraints. Under such constraints, the raw data $(X_1,Y_1),\ldots,(X_n,Y_n)$, taking values in $\mathbb{R}^d \times \mathbb{R}$, cannot be directly observed, and all estimators are functions of the randomised output from a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and here we add Laplace distributed noise to a discretisation of the location of a feature vector $X_i$ and to the value of its response variable $Y_i$. Based on this randomised data, we design a novel estimator of the regression function, which can be viewed as a privatised version of the well-studied partitioning regression estimator. The main result is that the estimator is strongly universally consistent. Our methods and analysis also give rise to a strongly universally consistent binary classification rule for locally differentially private data.