Equivalence between LINE and Matrix Factorization

TL;DR

This paper proves that the LINE network embedding method is mathematically equivalent to matrix factorization of specific PMI-based matrices, providing a theoretical foundation for understanding and extending LINE.

Contribution

It establishes that LINE's first- and second-order proximities correspond to factorizing distinct PMI matrices, linking network embedding to matrix factorization.

Findings

LINE(1st) factors a PMI-based matrix of vertex pairs in undirected networks.

LINE(2nd) factors a PMI-based matrix of vertex and context pairs in directed networks.

Theoretical connection between LINE and matrix factorization established.

Abstract

LINE [1], as an efficient network embedding method, has shown its effectiveness in dealing with large-scale undirected, directed, and/or weighted networks. Particularly, it proposes to preserve both the local structure (represented by First-order Proximity) and global structure (represented by Second-order Proximity) of the network. In this study, we prove that LINE with these two proximities (LINE(1st) and LINE(2nd)) are actually factoring two different matrices separately. Specifically, LINE(1st) is factoring a matrix M (1), whose entries are the doubled Pointwise Mutual Information (PMI) of vertex pairs in undirected networks, shifted by a constant. LINE(2nd) is factoring a matrix M (2), whose entries are the PMI of vertex and context pairs in directed networks, shifted by a constant. We hope this finding would provide a basis for further extensions and generalizations of LINE.