Equivalence Testing Under Privacy Constraints

TL;DR

This paper introduces a differentially private framework for equivalence testing of means and proportions, ensuring privacy without sacrificing statistical power, applicable in sensitive fields like healthcare.

Contribution

It develops DP-TOST, a unified simulation-based method for privacy-preserving equivalence testing with strong error control and practical applicability.

Findings

Maintains type-I error control at the nominal level.

Achieves power comparable to non-private tests with increased privacy budget or sample size.

Demonstrates effectiveness through simulations and real-world healthcare data.

Abstract

Protecting individual privacy is essential across research domains, from socio-economic surveys to big-tech user data. This need is particularly acute in healthcare, where analyses often involve sensitive patient information. A typical example is comparing treatment efficacy across hospitals or ensuring consistency in diagnostic laboratory calibrations, both requiring privacy-preserving statistical procedures. However, standard equivalence testing procedures for differences in proportions or means, commonly used to assess average equivalence, can inadvertently disclose sensitive information. To address this problem, we develop differentially private equivalence testing procedures that rely on simulation-based calibration, as the finite-sample distribution is analytically intractable. Our approach introduces a unified framework, termed DP-TOST, for conducting differentially private equivalence testing of both means and proportions. Through numerical simulations and real-world applications, we demonstrate that the proposed method maintains type-I error control at the nominal level and achieves power comparable to its non-private counterpart as the privacy budget and/or sample size increases, while ensuring strong privacy guarantees. These findings establish a reliable and practical framework for privacy-preserving equivalence testing in high-stakes fields such as healthcare, among others.