Truthful High Dimensional Sparse Linear Regression

TL;DR

This paper introduces a novel privacy-preserving mechanism for high-dimensional sparse linear regression that incentivizes truthful data reporting from strategic agents while maintaining estimation accuracy and low payment costs.

Contribution

It presents the first mechanism combining truthfulness, differential privacy, and high-dimensional sparse linear regression with a closed-form estimator.

Findings

Mechanism is $(o(1), O(n^{-rac{1}{3}}))$-jointly differentially private.

Achieves $o(1)$ error in estimating the underlying parameter.

Ensures a $(1-o(1))$-fraction of agents report truthfully in equilibrium.

Abstract

We study the problem of fitting the high dimensional sparse linear regression model with sub-Gaussian covariates and responses, where the data are provided by strategic or self-interested agents (individuals) who prioritize their privacy of data disclosure. In contrast to the classical setting, our focus is on designing mechanisms that can effectively incentivize most agents to truthfully report their data while preserving the privacy of individual reports. Simultaneously, we seek an estimator which should be close to the underlying parameter. We attempt to solve the problem by deriving a novel private estimator that has a closed-form expression. Based on the estimator, we propose a mechanism which has the following properties via some appropriate design of the computation and payment scheme: (1) the mechanism is $(o(1), O(n^{-\Omega({1})}))$-jointly differentially private, where $n$ is the number of agents; (2) it is an $o(\frac{1}{n})$-approximate Bayes Nash equilibrium for a $(1-o(1))$-fraction of agents to truthfully report their data; (3) the output could achieve an error of $o(1)$ to the underlying parameter; (4) it is individually rational for a $(1-o(1))$ fraction of agents in the mechanism; (5) the payment budget required from the analyst to run the mechanism is $o(1)$. To the best of our knowledge, this is the first study on designing truthful (and privacy-preserving) mechanisms for high dimensional sparse linear regression.