Convergent Privacy Framework for Multi-layer GNNs through Contractive Message Passing

TL;DR

This paper introduces CARIBOU, a privacy-preserving framework for multi-layer GNNs that leverages contractive message passing to ensure privacy budget convergence, improving privacy-utility trade-offs.

Contribution

It provides a theoretical analysis showing privacy budget convergence in multi-layer GNNs and proposes a contractive layer to enhance privacy guarantees while maintaining utility.

Findings

Privacy budget converges with the number of layers due to contractive properties.

CARIBOU outperforms existing methods in privacy-utility trade-offs.

Framework supports both training and inference with improved privacy auditing.

Abstract

Differential privacy (DP) has been integrated into graph neural networks (GNNs) to protect sensitive structural information, e.g., edges, nodes, and associated features across various applications. A prominent approach is to perturb the message-passing process, which forms the core of most GNN architectures. However, existing methods typically incur a privacy cost that grows linearly with the number of layers (e.g., GAP published in Usenix Security'23), ultimately requiring excessive noise to maintain a reasonable privacy level. This limitation becomes particularly problematic when multi-layer GNNs, which have shown better performance than one-layer GNN, are used to process graph data with sensitive information. In this paper, we theoretically establish that the privacy budget converges with respect to the number of layers by applying privacy amplification techniques to the message-passing process, exploiting the contractive properties inherent to standard GNN operations. Motivated by this analysis, we propose a simple yet effective Contractive Graph Layer (CGL) that ensures the contractiveness required for theoretical guarantees while preserving model utility. Our framework, CARIBOU, supports both training and inference, equipped with a contractive aggregation module, a privacy allocation module, and a privacy auditing module. Experimental evaluations demonstrate that CARIBOU significantly improves the privacy-utility trade-off and achieves superior performance in privacy auditing tasks.