Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant

TL;DR

This paper introduces a quadratic gradient method to improve privacy-preserving logistic regression training, achieving faster convergence and efficiency in encrypted data scenarios.

Contribution

It proposes a novel quadratic gradient variant that enhances existing optimization algorithms for privacy-preserving logistic regression, with demonstrated state-of-the-art results.

Findings

Achieves faster convergence rates than traditional methods.

Enables homomorphic logistic regression training in only four iterations.

Provides a unified framework combining first- and second-order optimization advantages.

Abstract

Training logistic regression over encrypted data has emerged as a prominent approach to addressing security concerns in recent years. In this paper, we introduce an efficient gradient variant, termed the \textit{quadratic gradient}, which is specifically designed for privacy-preserving logistic regression while remaining equally effective in plaintext optimization. By incorporating this quadratic gradient, we enhance Nesterov's Accelerated Gradient (NAG), Adaptive Gradient (AdaGrad), and Adam algorithms. We evaluate these enhanced algorithms across various datasets, with experimental results demonstrating state-of-the-art convergence rates that significantly outperform traditional first-order gradient methods. Furthermore, we apply the enhanced NAG method to implement homomorphic logistic regression training, achieving comparable performance within only four iterations. The proposed quadratic-gradient approach offers a unified framework that synergizes the advantages of first-order gradient methods and second-order Newton-type methods, suggesting broad applicability to diverse numerical optimization tasks.