Time-varying Graph Representation Learning via Higher-Order Skip-Gram with Negative Sampling

TL;DR

This paper introduces Higher-Order Skip-Gram with Negative Sampling (HOSGNS), a novel method for learning dynamic graph representations that effectively disentangle node and time roles, outperforming existing techniques in various tasks.

Contribution

The paper extends skip-gram models to tensor factorization for time-varying graphs, enabling efficient and effective dynamic network embeddings with fewer parameters.

Findings

HOSGNS outperforms state-of-the-art methods in network reconstruction.

HOSGNS accurately predicts disease spreading outcomes.

The approach efficiently disentangles node and time roles in dynamic graphs.

Abstract

Representation learning models for graphs are a successful family of techniques that project nodes into feature spaces that can be exploited by other machine learning algorithms. Since many real-world networks are inherently dynamic, with interactions among nodes changing over time, these techniques can be defined both for static and for time-varying graphs. Here, we build upon the fact that the skip-gram embedding approach implicitly performs a matrix factorization, and we extend it to perform implicit tensor factorization on different tensor representations of time-varying graphs. We show that higher-order skip-gram with negative sampling (HOSGNS) is able to disentangle the role of nodes and time, with a small fraction of the number of parameters needed by other approaches. We empirically evaluate our approach using time-resolved face-to-face proximity data, showing that the learned time-varying graph representations outperform state-of-the-art methods when used to solve downstream tasks such as network reconstruction, and to predict the outcome of dynamical processes such as disease spreading. The source code and data are publicly available at https://github.com/simonepiaggesi/hosgns.