Federated Graph Condensation with Information Bottleneck Principles

TL;DR

This paper introduces federated graph condensation (FGC), a privacy-preserving method for synthesizing small graphs from decentralized data, using information bottleneck principles to prevent data leakage while maintaining high performance.

Contribution

It proposes a novel federated framework for graph condensation that incorporates information bottleneck principles to protect privacy and achieves superior results compared to existing methods.

Findings

FGC effectively protects membership privacy during training.

The framework achieves comparable or better performance than centralized and federated baselines.

Incorporating information bottleneck principles enhances privacy without sacrificing accuracy.

Abstract

Graph condensation (GC), which reduces the size of a large-scale graph by synthesizing a small-scale condensed graph as its substitution, has benefited various graph learning tasks. However, existing GC methods rely on centralized data storage, which is unfeasible for real-world decentralized data distribution, and overlook data holders' privacy-preserving requirements. To bridge this gap, we propose and study the novel problem of federated graph condensation (FGC) for graph neural networks (GNNs). Specifically, we first propose a general framework for FGC, where we decouple the typical gradient matching process for GC into client-side gradient calculation and server-side gradient matching, integrating knowledge from multiple clients' subgraphs into one smaller condensed graph. Nevertheless, our empirical studies show that under the federated setting, the condensed graph will consistently leak data membership privacy, i.e., the condensed graph during federated training can be utilized to steal training data under the membership inference attack (MIA). To tackle this issue, we innovatively incorporate information bottleneck principles into the FGC, which only needs to extract partial node features in one local pre-training step and utilize the features during federated training. Theoretical and experimental analyses demonstrate that our framework consistently protects membership privacy during training. Meanwhile, it can achieve comparable and even superior performance against existing centralized GC and federated graph learning (FGL) methods.