SDP: Scalable Real-time Dynamic Graph Partitioner
Md Anwarul Kaium Patwary, Saurabh Garg, Sudheer Kumar Battula, Byeong, Kang

TL;DR
This paper introduces SDP, a scalable real-time dynamic graph partitioning method that reduces communication costs and balances load efficiently for evolving large graphs in streaming environments.
Contribution
The paper presents a novel streaming-based dynamic graph partitioning algorithm with new vertex assignment, communication-aware balancing, and scaling techniques.
Findings
Achieves up to 90% reduction in communication cost.
Balances load dynamically with 60-70% efficiency.
Reduces execution time significantly during partitioning.
Abstract
Time-evolving large graph has received attention due to their participation in real-world applications such as social networks and PageRank calculation. It is necessary to partition a large-scale dynamic graph in a streaming manner to overcome the memory bottleneck while partitioning the computational load. Reducing network communication and balancing the load between the partitions are the criteria to achieve effective run-time performance in graph partitioning. Moreover, an optimal resource allocation is needed to utilise the resources while catering the graph streams into the partitions. A number of existing partitioning algorithms (ADP, LogGP and LEOPARD) have been proposed to address the above problem. However, these partitioning methods are incapable of scaling the resources and handling the stream of data in real-time. In this study, we propose a dynamic graph partitioning…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Interconnection Networks and Systems · Advanced Graph Neural Networks
