Dynamic Embedding on Textual Networks via a Gaussian Process

TL;DR

This paper introduces DetGP, a novel end-to-end model for dynamic textual network embedding that efficiently updates node representations using Gaussian processes without re-training, improving link prediction and node classification.

Contribution

The paper proposes DetGP, a Gaussian process-based model that effectively handles dynamic graphs for textual network embedding without re-training, incorporating local and global structure modeling.

Findings

DetGP outperforms baseline methods in link prediction.

Efficiently updates embeddings for dynamic graphs.

Automatically learns the importance of local vs. global structure.

Abstract

Textual network embedding aims to learn low-dimensional representations of text-annotated nodes in a graph. Prior work in this area has typically focused on fixed graph structures; however, real-world networks are often dynamic. We address this challenge with a novel end-to-end node-embedding model, called Dynamic Embedding for Textual Networks with a Gaussian Process (DetGP). After training, DetGP can be applied efficiently to dynamic graphs without re-training or backpropagation. The learned representation of each node is a combination of textual and structural embeddings. Because the structure is allowed to be dynamic, our method uses the Gaussian process to take advantage of its non-parametric properties. To use both local and global graph structures, diffusion is used to model multiple hops between neighbors. The relative importance of global versus local structure for the embeddings is learned automatically. With the non-parametric nature of the Gaussian process, updating the embeddings for a changed graph structure requires only a forward pass through the learned model. Considering link prediction and node classification, experiments demonstrate the empirical effectiveness of our method compared to baseline approaches. We further show that DetGP can be straightforwardly and efficiently applied to dynamic textual networks.