Comprehend DeepWalk as Matrix Factorization

TL;DR

This paper reveals that DeepWalk, a popular node embedding algorithm for graphs, can be understood as matrix factorization of a specific matrix related to random walk probabilities, connecting it to word embedding techniques.

Contribution

It formally proves that DeepWalk is equivalent to factorizing a matrix derived from random walk probabilities, bridging graph embedding and matrix factorization methods.

Findings

DeepWalk factors a matrix of random walk probabilities.

The matrix M entries are logarithms of average walk probabilities.

This insight unifies DeepWalk with matrix factorization techniques.

Abstract

Word2vec, as an efficient tool for learning vector representation of words has shown its effectiveness in many natural language processing tasks. Mikolov et al. issued Skip-Gram and Negative Sampling model for developing this toolbox. Perozzi et al. introduced the Skip-Gram model into the study of social network for the first time, and designed an algorithm named DeepWalk for learning node embedding on a graph. We prove that the DeepWalk algorithm is actually factoring a matrix M where each entry M_{ij} is logarithm of the average probability that node i randomly walks to node j in fix steps.