PrivLogit: Efficient Privacy-preserving Logistic Regression by Tailoring Numerical Optimizers

TL;DR

This paper introduces PrivLogit, a privacy-preserving logistic regression method that customizes numerical optimization for secure settings, achieving significant speedups without sacrificing accuracy or privacy.

Contribution

It proposes a novel optimization approach tailored for cryptographic secure computing, enabling faster privacy-preserving logistic regression protocols.

Findings

Speedup up to 2.3x and 8.1x over state-of-the-art methods

Maintains accuracy and privacy in large-scale distributed settings

Faster performance as data size increases

Abstract

Safeguarding privacy in machine learning is highly desirable, especially in collaborative studies across many organizations. Privacy-preserving distributed machine learning (based on cryptography) is popular to solve the problem. However, existing cryptographic protocols still incur excess computational overhead. Here, we make a novel observation that this is partially due to naive adoption of mainstream numerical optimization (e.g., Newton method) and failing to tailor for secure computing. This work presents a contrasting perspective: customizing numerical optimization specifically for secure settings. We propose a seemingly less-favorable optimization method that can in fact significantly accelerate privacy-preserving logistic regression. Leveraging this new method, we propose two new secure protocols for conducting logistic regression in a privacy-preserving and distributed manner. Extensive theoretical and empirical evaluations prove the competitive performance of our two secure proposals while without compromising accuracy or privacy: with speedup up to 2.3x and 8.1x, respectively, over state-of-the-art; and even faster as data scales up. Such drastic speedup is on top of and in addition to performance improvements from existing (and future) state-of-the-art cryptography. Our work provides a new way towards efficient and practical privacy-preserving logistic regression for large-scale studies which are common for modern science.