COSINE: Compressive Network Embedding on Large-scale Information Networks

TL;DR

COSINE introduces a memory-efficient network embedding method that shares parameters among similar nodes, significantly improving training speed and accuracy on large-scale networks.

Contribution

The paper proposes COSINE, a novel algorithm that reduces memory usage and accelerates training by parameter sharing based on graph partitioning, applicable to any embedding method.

Findings

Up to 23% improvement in classification accuracy.

Up to 25% improvement in link prediction.

Training time reduced by 30% to 70%.

Abstract

There is recently a surge in approaches that learn low-dimensional embeddings of nodes in networks. As there are many large-scale real-world networks, it's inefficient for existing approaches to store amounts of parameters in memory and update them edge after edge. With the knowledge that nodes having similar neighborhood will be close to each other in embedding space, we propose COSINE (COmpresSIve NE) algorithm which reduces the memory footprint and accelerates the training process by parameters sharing among similar nodes. COSINE applies graph partitioning algorithms to networks and builds parameter sharing dependency of nodes based on the result of partitioning. With parameters sharing among similar nodes, COSINE injects prior knowledge about higher structural information into training process which makes network embedding more efficient and effective. COSINE can be applied to any embedding lookup method and learn high-quality embeddings with limited memory and shorter training time. We conduct experiments of multi-label classification and link prediction, where baselines and our model have the same memory usage. Experimental results show that COSINE gives baselines up to 23% increase on classification and up to 25% increase on link prediction. Moreover, time of all representation learning methods using COSINE decreases from 30% to 70%.