Independent Distribution Regularization for Private Graph Embedding

TL;DR

This paper introduces PVGAE, a privacy-preserving graph embedding method that uses independent distribution regularization to improve utility and privacy, addressing limitations of existing adversarial approaches.

Contribution

The paper proposes PVGAE with a novel regularization enforcing encoder independence, enhancing privacy and utility without requiring prior sensitive attribute access.

Findings

PVGAE outperforms baselines in utility and privacy on real datasets.

The regularization effectively enforces encoder independence from mutual information perspective.

Experimental results validate the theoretical advantages of the proposed method.

Abstract

Learning graph embeddings is a crucial task in graph mining tasks. An effective graph embedding model can learn low-dimensional representations from graph-structured data for data publishing benefiting various downstream applications such as node classification, link prediction, etc. However, recent studies have revealed that graph embeddings are susceptible to attribute inference attacks, which allow attackers to infer private node attributes from the learned graph embeddings. To address these concerns, privacy-preserving graph embedding methods have emerged, aiming to simultaneously consider primary learning and privacy protection through adversarial learning. However, most existing methods assume that representation models have access to all sensitive attributes in advance during the training stage, which is not always the case due to diverse privacy preferences. Furthermore, the commonly used adversarial learning technique in privacy-preserving representation learning suffers from unstable training issues. In this paper, we propose a novel approach called Private Variational Graph AutoEncoders (PVGAE) with the aid of independent distribution penalty as a regularization term. Specifically, we split the original variational graph autoencoder (VGAE) to learn sensitive and non-sensitive latent representations using two sets of encoders. Additionally, we introduce a novel regularization to enforce the independence of the encoders. We prove the theoretical effectiveness of regularization from the perspective of mutual information. Experimental results on three real-world datasets demonstrate that PVGAE outperforms other baselines in private embedding learning regarding utility performance and privacy protection.