Free Energy Node Embedding via Generalized Skip-gram with Negative Sampling

TL;DR

This paper introduces a flexible node embedding method using free energy distances and a generalized skip-gram matrix factorization, improving the preservation of high-similarity node pairs and leveraging GPU efficiency.

Contribution

It proposes a novel similarity measure based on free energy distance and a generalized matrix factorization approach for node embedding, enhancing existing unsupervised methods.

Findings

Improved node clustering, classification, and link prediction results.

Better preservation of high-similarity node pairs.

Efficient GPU implementation of the proposed method.

Abstract

A widely established set of unsupervised node embedding methods can be interpreted as consisting of two distinctive steps: i) the definition of a similarity matrix based on the graph of interest followed by ii) an explicit or implicit factorization of such matrix. Inspired by this viewpoint, we propose improvements in both steps of the framework. On the one hand, we propose to encode node similarities based on the free energy distance, which interpolates between the shortest path and the commute time distances, thus, providing an additional degree of flexibility. On the other hand, we propose a matrix factorization method based on a loss function that generalizes that of the skip-gram model with negative sampling to arbitrary similarity matrices. Compared with factorizations based on the widely used $\ell_2$ loss, the proposed method can better preserve node pairs associated with higher similarity scores. Moreover, it can be easily implemented using advanced automatic differentiation toolkits and computed efficiently by leveraging GPU resources. Node clustering, node classification, and link prediction experiments on real-world datasets demonstrate the effectiveness of incorporating free-energy-based similarities as well as the proposed matrix factorization compared with state-of-the-art alternatives.