Training Differentially Private Models with Secure Multiparty Computation

TL;DR

This paper introduces a novel MPC-based method for training differentially private machine learning models, achieving higher accuracy than traditional DP methods while maintaining formal privacy guarantees.

Contribution

It presents a combined MPC and DP protocol for training privacy-preserving models without accuracy loss, advancing secure multi-party machine learning.

Findings

Achieved higher accuracy than pure DP approaches.

Maintained formal privacy guarantees.

Won first place in iDASH2021 competition.

Abstract

We address the problem of learning a machine learning model from training data that originates at multiple data owners while providing formal privacy guarantees regarding the protection of each owner's data. Existing solutions based on Differential Privacy (DP) achieve this at the cost of a drop in accuracy. Solutions based on Secure Multiparty Computation (MPC) do not incur such accuracy loss but leak information when the trained model is made publicly available. We propose an MPC solution for training DP models. Our solution relies on an MPC protocol for model training, and an MPC protocol for perturbing the trained model coefficients with Laplace noise in a privacy-preserving manner. The resulting MPC+DP approach achieves higher accuracy than a pure DP approach while providing the same formal privacy guarantees. Our work obtained first place in the iDASH2021 Track III competition on confidential computing for secure genome analysis.