Every Query Counts: Analyzing the Privacy Loss of Exploratory Data Analyses

TL;DR

This paper quantifies the privacy loss incurred during exploratory data analysis, emphasizing its significance in the overall privacy budget for machine learning workflows.

Contribution

It provides a detailed analysis of privacy loss for common statistical functions used in exploratory data analysis, highlighting the need to include this in privacy accounting.

Findings

Quantifies privacy loss for basic statistical functions.

Shows importance of accounting for privacy loss in exploratory analysis.

Highlights potential underestimation of privacy risk in current practices.

Abstract

An exploratory data analysis is an essential step for every data analyst to gain insights, evaluate data quality and (if required) select a machine learning model for further processing. While privacy-preserving machine learning is on the rise, more often than not this initial analysis is not counted towards the privacy budget. In this paper, we quantify the privacy loss for basic statistical functions and highlight the importance of taking it into account when calculating the privacy-loss budget of a machine learning approach.