Differentially Private Two-Stage Empirical Risk Minimization and Applications to Individualized Treatment Rule

TL;DR

This paper introduces DP-2ERM, a novel framework for differentially private two-stage empirical risk minimization that improves utility in individualized treatment rule applications by carefully calibrating noise injection.

Contribution

The paper proposes DP-2ERM, a method that injects noise only into the second stage of a two-stage process, preserving first stage weights and improving privacy-utility trade-offs.

Findings

DP-2ERM outperforms existing methods in utility while maintaining privacy guarantees.

Theoretical bounds on weight perturbations and estimator sensitivity are established.

Real-world ITR applications demonstrate significant utility improvements.

Abstract

Differential Privacy (DP) provides a rigorous framework for deriving privacy-preserving estimators by injecting calibrated noise to mask individual contributions while preserving population-level insights. Its central challenge lies in the privacy-utility trade-off: calibrating noise levels to ensure robust protection without compromising statistical performance. Standard DP methods struggle with a particular class of two-stage problems prevalent in individualized treatment rules (ITRs) and causal inference. In these settings, data-dependent weights are first computed to satisfy distributional constraints, such as covariate balance, before the final parameter of interest is estimated. Current DP approaches often privatize stages independently, which either degrades weight efficacy-leading to biased and inconsistent estimates-or introduces excessive noise to account for worst-case scenarios. To address these challenges, we propose the Differentially Private Two-Stage Empirical Risk Minimization (DP-2ERM), a framework that injects a carefully calibrated noise only into the second stage while maintaining privacy for the entire pipeline and preserving the integrity of the first stage weights. Our theoretical contributions include deterministic bounds on weight perturbations across various widely used weighting methods, and probabilistic bounds on sensitivity for the final estimator. Simulations and real-world applications in ITR demonstrate that DP-2ERM significantly enhances utility over existing methods while providing rigorous privacy guarantees.