Neural Latent Space Model for Dynamic Networks and Temporal Knowledge Graphs

TL;DR

This paper introduces a neural latent space model with variational inference for dynamic networks, capturing temporal evolution and multiple relations, and demonstrating superior link prediction performance on various real-world networks.

Contribution

It presents a unified probabilistic model for both homogeneous and heterogeneous dynamic networks, incorporating temporal evolution of node embeddings and interaction matrices.

Findings

Significantly outperforms existing models in link forecasting.

Effectively models multiple relation types in heterogeneous networks.

Provides interpretable latent representations of network evolution.

Abstract

Although static networks have been extensively studied in machine learning, data mining, and AI communities for many decades, the study of dynamic networks has recently taken center stage due to the prominence of social media and its effects on the dynamics of social networks. In this paper, we propose a statistical model for dynamically evolving networks, together with a variational inference approach. Our model, Neural Latent Space Model with Variational Inference, encodes edge dependencies across different time snapshots. It represents nodes via latent vectors and uses interaction matrices to model the presence of edges. These matrices can be used to incorporate multiple relations in heterogeneous networks by having a separate matrix for each of the relations. To capture the temporal dynamics, both node vectors and interaction matrices are allowed to evolve with time. Existing network analysis methods use representation learning techniques for modelling networks. These techniques are different for homogeneous and heterogeneous networks because heterogeneous networks can have multiple types of edges and nodes as opposed to a homogeneous network. Unlike these, we propose a unified model for homogeneous and heterogeneous networks in a variational inference framework. Moreover, the learned node latent vectors and interaction matrices may be interpretable and therefore provide insights on the mechanisms behind network evolution. We experimented with a single step and multi-step link forecasting on real-world networks of homogeneous, bipartite, and heterogeneous nature, and demonstrated that our model significantly outperforms existing models.