The Gaussian Mixing Mechanism: Renyi Differential Privacy via Gaussian Sketches

TL;DR

This paper introduces a refined analysis of Gaussian sketching under Renyi Differential Privacy, leading to tighter privacy bounds and improved utility and efficiency in linear regression tasks.

Contribution

It provides a novel Renyi Differential Privacy analysis for Gaussian sketching, resulting in tighter privacy bounds and enhanced performance guarantees.

Findings

Tighter privacy bounds under RDP for Gaussian sketching.

Improved utility guarantees in linear regression.

Empirical performance gains and reduced runtime on multiple datasets.

Abstract

Gaussian sketching, which consists of pre-multiplying the data with a random Gaussian matrix, is a widely used technique for multiple problems in data science and machine learning, with applications spanning computationally efficient optimization, coded computing, and federated learning. This operation also provides differential privacy guarantees due to its inherent randomness. In this work, we revisit this operation through the lens of Renyi Differential Privacy (RDP), providing a refined privacy analysis that yields significantly tighter bounds than prior results. We then demonstrate how this improved analysis leads to performance improvement in different linear regression settings, establishing theoretical utility guarantees. Empirically, our methods improve performance across multiple datasets and, in several cases, reduce runtime.