Privacy-Preserving Representation Learning for Text-Attributed Networks with Simplicial Complexes

TL;DR

This paper explores privacy-preserving methods for learning network representations in text-attributed networks using simplicial neural networks, addressing higher-order interactions and potential privacy vulnerabilities.

Contribution

It introduces a framework for learning representations with SNNs in text-attributed networks and investigates privacy attacks and defenses specific to topological data analysis.

Findings

Analysis of vulnerability of SNNs to membership inference attacks

Development of a differentially private learning algorithm for SNNs

Enhanced privacy protection for higher-order network representations

Abstract

Although recent network representation learning (NRL) works in text-attributed networks demonstrated superior performance for various graph inference tasks, learning network representations could always raise privacy concerns when nodes represent people or human-related variables. Moreover, standard NRLs that leverage structural information from a graph proceed by first encoding pairwise relationships into learned representations and then analysing its properties. This approach is fundamentally misaligned with problems where the relationships involve multiple points, and topological structure must be encoded beyond pairwise interactions. Fortunately, the machinery of topological data analysis (TDA) and, in particular, simplicial neural networks (SNNs) offer a mathematically rigorous framework to learn higher-order interactions between nodes. It is critical to investigate if the representation outputs from SNNs are more vulnerable compared to regular representation outputs from graph neural networks (GNNs) via pairwise interactions. In my dissertation, I will first study learning the representations with text attributes for simplicial complexes (RT4SC) via SNNs. Then, I will conduct research on two potential attacks on the representation outputs from SNNs: (1) membership inference attack, which infers whether a certain node of a graph is inside the training data of the GNN model; and (2) graph reconstruction attacks, which infer the confidential edges of a text-attributed network. Finally, I will study a privacy-preserving deterministic differentially private alternating direction method of multiplier to learn secure representation outputs from SNNs that capture multi-scale relationships and facilitate the passage from local structure to global invariant features on text-attributed networks.