KV-Auditor: Auditing Local Differential Privacy for Correlated Key-Value Estimation

TL;DR

KV-Auditor is a novel framework that empirically audits local differential privacy mechanisms for correlated key-value data, addressing a gap in existing methods by handling continuous data and complex mechanisms.

Contribution

The paper introduces KV-Auditor, the first comprehensive auditing framework for LDP key-value estimation, capable of analyzing both discrete and continuous data in interactive and non-interactive settings.

Findings

KV-Auditor accurately estimates empirical privacy bounds.

It effectively handles large and small domain scenarios.

Experimental results validate its robustness and practical utility.

Abstract

To protect privacy for data-collection-based services, local differential privacy (LDP) is widely adopted due to its rigorous theoretical bound on privacy loss. However, mistakes in complex theoretical analysis or subtle implementation errors may undermine its practical guarantee. To address this, auditing is crucial to confirm that LDP protocols truly protect user data. However, existing auditing methods, though, mainly target machine learning and federated learning tasks based on centralized differentially privacy (DP), with limited attention to LDP. Moreover, the few studies on LDP auditing focus solely on simple frequency estimation task for discrete data, leaving correlated key-value data - which requires both discrete frequency estimation for keys and continuous mean estimation for values - unexplored. To bridge this gap, we propose KV-Auditor, a framework for auditing LDP-based key-value estimation mechanisms by estimating their empirical privacy lower bounds. Rather than traditional LDP auditing methods that relies on binary output predictions, KV-Auditor estimates this lower bound by analyzing unbounded output distributions, supporting continuous data. Specifically, we classify state-of-the-art LDP key-value mechanisms into interactive and non-interactive types. For non-interactive mechanisms, we propose horizontal KV-Auditor for small domains with sufficient samples and vertical KV-Auditor for large domains with limited samples. For interactive mechanisms, we design a segmentation strategy to capture incremental privacy leakage across iterations. Finally, we perform extensive experiments to validate the effectiveness of our approach, offering insights for optimizing LDP-based key-value estimators.