Privacy-Preserving Distributed Stochastic Optimization with Homomorphic Encryption and Heterogeneous Stepsizes

TL;DR

This paper introduces a privacy-preserving distributed stochastic gradient descent algorithm that combines homomorphic encryption with heterogeneous stepsizes, ensuring convergence and privacy without trusted neighbors.

Contribution

It presents a novel algorithm integrating Paillier encryption with adaptive stepsizes, providing privacy and convergence guarantees in distributed stochastic optimization.

Findings

Ensures privacy against internal and external threats without trusted neighbors.

Achieves almost sure convergence to the optimal solution.

Demonstrates effectiveness and efficiency through numerical simulations.

Abstract

Distributed stochastic optimization enables multi-agent collaboration in applications such as distributed learning and sensor networks, but also raises critical privacy concerns due to the involvement of sensitive data. While existing privacy-preserving approaches often face limitations in balancing accuracy with efficiency, we propose a novel distributed stochastic gradient descent algorithm that integrates Paillier homomorphic encryption with heterogeneous and time-varying random stepsizes. The proposed algorithm provides inherent privacy protection against both internal honest-but-curious agents and external eavesdroppers, without relying on any trusted neighbors. Furthermore, we incorporate an attenuation factor to effectively mitigate quantization error induced by the encryption process, ensuring almost sure convergence to the optimal solution while maintaining privacy preservation. Numerical simulations demonstrate the effectiveness and efficiency of the proposed approach.