SketchNE: Embedding Billion-Scale Networks Accurately in One Hour

TL;DR

SketchNE is a scalable, memory-efficient network embedding method capable of accurately embedding billion-scale networks within one hour on a CPU-only machine, outperforming existing methods in effectiveness and efficiency.

Contribution

It introduces SketchNE, a novel approach that avoids explicit matrix construction and factorization, enabling high-quality embeddings for extremely large networks on limited hardware.

Findings

Successfully embedded networks with over 3.5 billion vertices and 225 billion edges in one hour.

Outperformed state-of-the-art methods in vertex classification and link prediction tasks.

Achieved 282% relative improvement in HITS@10 over LightNE.

Abstract

We study large-scale network embedding with the goal of generating high-quality embeddings for networks with more than 1 billion vertices and 100 billion edges. Recent attempts LightNE and NetSMF propose to sparsify and factorize the (dense) NetMF matrix for embedding large networks, where NetMF is a theoretically-grounded network embedding method. However, there is a trade-off between their embeddings' quality and scalability due to their expensive memory requirements, making embeddings less effective under real-world memory constraints. Therefore, we present the SketchNE model, a scalable, effective, and memory-efficient network embedding solution developed for a single machine with CPU only. The main idea of SketchNE is to avoid the explicit construction and factorization of the NetMF matrix either sparsely or densely when producing the embeddings through the proposed sparse-sign randomized single-pass SVD algorithm. We conduct extensive experiments on nine datasets of various sizes for vertex classification and link prediction, demonstrating the consistent outperformance of SketchNE over state-of-the-art baselines in terms of both effectiveness and efficiency. SketchNE costs only 1.0 hours to embed the Hyperlink2012 network with 3.5 billion vertices and 225 billion edges on a CPU-only single machine with embedding superiority (e.g., a 282% relative HITS@10 gain over LightNE).