Privacy-Preserving Distributed Expectation Maximization for Gaussian Mixture Model using Subspace Perturbation

TL;DR

This paper analyzes privacy risks in federated EM algorithms for Gaussian mixture models and proposes a decentralized, privacy-preserving method using subspace perturbation, enhancing privacy without sacrificing accuracy.

Contribution

It introduces a novel decentralized privacy-preserving approach for federated EM, addressing privacy leakage and defending against specific adversary models.

Findings

The proposed method reduces privacy leakage compared to existing approaches.

It maintains high accuracy while ensuring privacy in federated learning.

Numerical results demonstrate superior privacy and performance performance.

Abstract

Privacy has become a major concern in machine learning. In fact, the federated learning is motivated by the privacy concern as it does not allow to transmit the private data but only intermediate updates. However, federated learning does not always guarantee privacy-preservation as the intermediate updates may also reveal sensitive information. In this paper, we give an explicit information-theoretical analysis of a federated expectation maximization algorithm for Gaussian mixture model and prove that the intermediate updates can cause severe privacy leakage. To address the privacy issue, we propose a fully decentralized privacy-preserving solution, which is able to securely compute the updates in each maximization step. Additionally, we consider two different types of security attacks: the honest-but-curious and eavesdropping adversary models. Numerical validation shows that the proposed approach has superior performance compared to the existing approach in terms of both the accuracy and privacy level.