UniGPS: A Unified Programming Framework for Distributed Graph Processing

TL;DR

UniGPS introduces a unified, cross-platform graph programming framework that simplifies distributed graph processing for users by supporting multiple models and languages, enabling scalable processing without extensive platform knowledge.

Contribution

The paper presents UniGPS, a unified framework with a cross-platform programming model VCProg and Python support, reducing learning and migration costs in distributed graph processing.

Findings

Supports processing graphs beyond single-machine memory limits.

Achieves near-linear data scalability.

Demonstrates compatibility with popular graph models.

Abstract

The industry and academia have proposed many distributed graph processing systems. However, the existing systems are not friendly enough for users like data analysts and algorithm engineers. On the one hand, the programing models and interfaces differ a lot in the existing systems, leading to high learning costs and program migration costs. On the other hand, these graph processing systems are tightly bound to the underlying distributed computing platforms, requiring users to be familiar with distributed computing. To improve the usability of distributed graph processing, we propose a unified distributed graph programming framework UniGPS. Firstly, we propose a unified cross-platform graph programming model VCProg for UniGPS. VCProg hides details of distributed computing from users. It is compatible with the popular graph programming models Pregel, GAS, and Push-Pull. VCProg programs can be executed by compatible distributed graph processing systems without modification, reducing the learning overheads of users. Secondly, UniGPS supports Python as the programming language. We propose an interprocess-communication-based execution environment isolation mechanism to enable Java/C++-based systems to call user-defined methods written in Python. The experimental results show that UniGPS enables users to process big graphs beyond the memory capacity of a single machine without sacrificing usability. UniGPS shows near-linear data scalability and machine scalability.