Privacy-Preserving Dataset Combination

TL;DR

This paper introduces { exttt{SecureKL}}, a privacy-preserving protocol for evaluating dataset utility internally without privacy leakage, enabling secure data sharing decisions in sensitive domains like healthcare.

Contribution

The paper presents { exttt{SecureKL}}, the first secure protocol for dataset-to-dataset evaluation with zero privacy leakage, facilitating privacy-aware data sharing decisions.

Findings

Achieves over 90% correlation with non-private evaluations.

Effectively identifies beneficial data collaborations in heterogeneous domains.

Outperforms privacy-agnostic utility assessments that leak information.

Abstract

Access to diverse, high-quality datasets is crucial for machine learning model performance, yet data sharing remains limited by privacy concerns and competitive interests, particularly in regulated domains like healthcare. This dynamic especially disadvantages smaller organizations that lack resources to purchase data or negotiate favorable sharing agreements, due to the inability to \emph{privately} assess external data's utility. To resolve privacy and uncertainty tensions simultaneously, we introduce {\SecureKL}, the first secure protocol for dataset-to-dataset evaluations with zero privacy leakage, designed to be applied preceding data sharing. {\SecureKL} evaluates a source dataset against candidates, performing dataset divergence metrics internally with private computations, all without assuming downstream models. On real-world data, {\SecureKL} achieves high consistency ($>90\%$ correlation with non-private counterparts) and successfully identifies beneficial data collaborations in highly-heterogeneous domains (ICU mortality prediction across hospitals and income prediction across states). Our results highlight that secure computation maximizes data utilization, outperforming privacy-agnostic utility assessments that leak information.