$(\epsilon, \delta)$-Differentially Private Partial Least Squares Regression

TL;DR

This paper introduces a differentially private version of Partial Least Squares (PLS) regression, called edPLS, which adds calibrated Gaussian noise to key model outputs to protect sensitive data while maintaining predictive accuracy.

Contribution

The paper develops a novel $(\epsilon,\delta)$-differentially private PLS algorithm by integrating Gaussian noise based on sensitivity analysis, ensuring data privacy in multivariate calibration.

Findings

edPLS effectively prevents privacy attacks.

Predictive performance remains competitive at strong privacy levels.

The method is practical for privacy-preserving multivariate analysis.

Abstract

As data-privacy requirements are becoming increasingly stringent and statistical models based on sensitive data are being deployed and used more routinely, protecting data-privacy becomes pivotal. Partial Least Squares (PLS) regression is the premier tool for building such models in analytical chemistry, yet it does not inherently provide privacy guarantees, leaving sensitive (training) data vulnerable to privacy attacks. To address this gap, we propose an $(\epsilon, \delta)$-differentially private PLS (edPLS) algorithm, which integrates well-studied and theoretically motivated Gaussian noise-adding mechanisms into the PLS algorithm to ensure the privacy of the data underlying the model. Our approach involves adding carefully calibrated Gaussian noise to the outputs of four key functions in the PLS algorithm: the weights, scores, $X$-loadings, and $Y$-loadings. The noise variance is determined based on the global sensitivity of each function, ensuring that the privacy loss is controlled according to the $(\epsilon, \delta)$-differential privacy framework. Specifically, we derive the sensitivity bounds for each function and use these bounds to calibrate the noise added to the model components. Experimental results demonstrate that edPLS effectively renders privacy attacks, aimed at recovering unique sources of variability in the training data, ineffective. Application of edPLS to the NIR corn benchmark dataset shows that the root mean squared error of prediction (RMSEP) remains competitive even at strong privacy levels (i.e., $\epsilon=1$), given proper pre-processing of the corresponding spectra. These findings highlight the practical utility of edPLS in creating privacy-preserving multivariate calibrations and for the analysis of their privacy-utility trade-offs.