Global Vectors for Node Representations

TL;DR

This paper introduces a GloVe-inspired matrix factorization method for network embedding that effectively handles non co-occurrence data, extends to document networks, and achieves state-of-the-art results with stable hyper-parameter sensitivity.

Contribution

It proposes a novel GloVe-based matrix factorization approach for network embedding, extending to document networks, with improved performance and robustness.

Findings

Achieves state-of-the-art performance in network embedding tasks.

Handles non co-occurrence data more effectively than SGNS.

Provides tools for exploring document networks with combined content and network insights.

Abstract

Most network embedding algorithms consist in measuring co-occurrences of nodes via random walks then learning the embeddings using Skip-Gram with Negative Sampling. While it has proven to be a relevant choice, there are alternatives, such as GloVe, which has not been investigated yet for network embedding. Even though SGNS better handles non co-occurrence than GloVe, it has a worse time-complexity. In this paper, we propose a matrix factorization approach for network embedding, inspired by GloVe, that better handles non co-occurrence with a competitive time-complexity. We also show how to extend this model to deal with networks where nodes are documents, by simultaneously learning word, node and document representations. Quantitative evaluations show that our model achieves state-of-the-art performance, while not being so sensitive to the choice of hyper-parameters. Qualitatively speaking, we show how our model helps exploring a network of documents by generating complementary network-oriented and content-oriented keywords.