A New Perspective of Graph Data and A Generic and Efficient Method for Large Scale Graph Data Traversal

TL;DR

This paper introduces a new perspective on graph data, classifies algorithms based on data correlation, and proposes an efficient method for large-scale graph traversal that significantly improves performance across multiple platforms.

Contribution

It presents a novel classification of graph algorithms and a general method for processing large-scale graph data by separating low-degree vertices and core subgraphs.

Findings

Achieved up to 31.8% performance improvement on major platforms.

Reduced random memory access, enhancing efficiency.

Ranked among the Green graph500 in November 2019.

Abstract

The BFS algorithm is a basic graph data processing algorithm and many other graph data processing algorithms have similar architectural features with BFS algorithm and can be built on the basis of BFS algorithm model. We analyze the differences between graph algorithms and traditional high-performance algorithms in detail, propose a new way of classifying algorithms into data independent algorithm and data correlation algorithm based on their run-time correlation with data, and use this new classification to explain the validity of the methods proposed in this paper. Through a deeper analysis of graph data, we propose a new fundamental perspective on understanding graph data, establishing a link between two basic data structures, graph and tree, and viewing graph data as consisting of smaller subgraphs and edge trees. Small degree vertices are found to be one of important cause of random memory access. Based on this, we propose a general, easy to implement, and efficient method for graph data processing, with the basic idea of treating low-degree vertices and core subgraphs separately, thus significantly reducing the size of random memory access and improving the efficiency of memory access. Finally, we evaluated the performance of the method on three major data center computing platforms (Intel, AMD, and ARM), and the experiments showed that it brought 19.7%, 31.8% and 17.9% performance improvement, respectively, with a performance-power ratio of 282.70 MTEPS/s on the ARM platform, ranking it among the Green graph500 in November 2019. World No. 1 on the big dataset list.