GCON: Differentially Private Graph Convolutional Network via Objective Perturbation

TL;DR

GCON introduces a novel method for training Graph Convolutional Networks with differential privacy by perturbing the objective function, achieving strong privacy guarantees while maintaining high model utility across various datasets.

Contribution

GCON is the first to apply objective perturbation for differential privacy in GCNs, avoiding distortions caused by previous methods and providing tight sensitivity bounds.

Findings

GCON outperforms existing DP methods on benchmark datasets.

Theoretical bounds on sensitivity are tight and practical.

GCON maintains high utility with strong privacy guarantees.

Abstract

Graph Convolutional Networks (GCNs) are a popular machine learning model with a wide range of applications in graph analytics, including healthcare, transportation, and finance. However, a GCN trained without privacy protection measures may memorize private interpersonal relationships in the training data through its model parameters. This poses a substantial risk of compromising privacy through link attacks, potentially leading to violations of privacy regulations such as GDPR. To defend against such attacks, a promising approach is to train the GCN with differential privacy (DP), a rigorous framework that provides strong privacy protection by injecting random noise into the training process. However, training a GCN under DP is a highly challenging task. Existing solutions either perturb the graph topology or inject randomness into the graph convolution operations, or overestimate the amount of noise required, resulting in severe distortions of the network's message aggregation and, thus, poor model utility. Motivated by this, we propose GCON, a novel and effective solution for training GCNs with edge differential privacy. GCON leverages the classic idea of perturbing the objective function to satisfy DP and maintains an unaltered graph convolution process. Our rigorous theoretical analysis offers tight, closed-form bounds on the sensitivity of the graph convolution results and quantifies the impact of an edge modification on the trained model parameters. Extensive experiments using multiple benchmark datasets across diverse settings demonstrate the consistent superiority of GCON over existing solutions.