Maiter: An Asynchronous Graph Processing Framework for Delta-based Accumulative Iterative Computation
Yanfeng Zhang, Qixin Gao, Lixin Gao, Cuirong Wang

TL;DR
Maiter is a novel asynchronous graph processing framework that leverages delta-based accumulative iterative computation to significantly accelerate large-scale graph algorithms by processing only changes between iterations.
Contribution
The paper introduces DAIC and its implementation in Maiter, enabling asynchronous, efficient, and scalable graph processing for iterative algorithms in distributed environments.
Findings
Maiter achieves up to 60x speedup over Hadoop.
Maiter outperforms existing state-of-the-art frameworks.
DAIC reduces unnecessary computations by focusing on changes.
Abstract
Myriad of graph-based algorithms in machine learning and data mining require parsing relational data iteratively. These algorithms are implemented in a large-scale distributed environment in order to scale to massive data sets. To accelerate these large-scale graph-based iterative computations, we propose delta-based accumulative iterative computation (DAIC). Different from traditional iterative computations, which iteratively update the result based on the result from the previous iteration, DAIC updates the result by accumulating the "changes" between iterations. By DAIC, we can process only the "changes" to avoid the negligible updates. Furthermore, we can perform DAIC asynchronously to bypass the high-cost synchronous barriers in heterogeneous distributed environments. Based on the DAIC model, we design and implement an asynchronous graph processing framework, Maiter. We evaluate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Advanced Graph Neural Networks · Data Management and Algorithms
