Scalable Graph Embedding LearningOn A Single GPU

TL;DR

This paper presents a hybrid CPU-GPU framework for scalable graph embedding learning that can handle large-scale graphs beyond single machine memory limits, improving performance and accuracy.

Contribution

It introduces a novel hybrid CPU-GPU system enabling large-scale graph embedding training on a single GPU, surpassing existing systems in scalability and efficiency.

Findings

Scales training to datasets much larger than single GPU memory.

Outperforms existing systems on benchmark datasets.

Embeddings improve downstream task performance.

Abstract

Graph embedding techniques have attracted growing interest since they convert the graph data into continuous and low-dimensional space. Effective graph analytic provides users a deeper understanding of what is behind the data and thus can benefit a variety of machine learning tasks. With the current scale of real-world applications, most graph analytic methods suffer high computation and space costs. These methods and systems can process a network with thousands to a few million nodes. However, scaling to large-scale networks remains a challenge. The complexity of training graph embedding system requires the use of existing accelerators such as GPU. In this paper, we introduce a hybrid CPU-GPU framework that addresses the challenges of learning embedding of large-scale graphs. The performance of our method is compared qualitatively and quantitatively with the existing embedding systems on common benchmarks. We also show that our system can scale training to datasets with an order of magnitude greater than a single machine's total memory capacity. The effectiveness of the learned embedding is evaluated within multiple downstream applications. The experimental results indicate the effectiveness of the learned embedding in terms of performance and accuracy.