Computation of K-Core Decomposition on Giraph

TL;DR

This paper compares distributed Giraph and single-machine GraphChi implementations of k-core decomposition, finding Giraph more efficient for large graphs but less so for small ones.

Contribution

It presents a distributed Giraph algorithm for k-core decomposition and compares its performance with GraphChi on large networks.

Findings

Giraph outperforms GraphChi on large datasets

Giraph's communication overhead affects small dataset performance

Distributed approach benefits large-scale social network analysis

Abstract

Graphs are an essential data structure that can represent the structure of social networks. Many online companies, in order to provide intelligent and personalized services for their users, aim to comprehensively analyze a significant amount of graph data with different features. One example is k-core decomposition which captures the degree of connectedness in social graphs. The main purpose of this report is to explore a distributed algorithm for k-core decomposition on Apache Giraph. Namely, we would like to determine whether a cluster-based, Giraph implementation of k-core decomposition that we provide is more efficient than a single-machine, disk-based implementation on GraphChi for large networks. In this report, we describe (a) the programming model of Giraph and GraphChi, (b) the specific implementation of k-core decomposition with Giraph, and (c) the result comparison between Giraph and GraphChi. By analyzing the results, we conclude that Giraph is faster than GraphChi when dealing with large data. However, since worker nodes need time to communicate with each other, Giraph is not very efficient for small data.