StarDist: A Code Generator for Distributed Graph Algorithms
Barenya Kumar Nandy, Rupesh Nasre

TL;DR
StarDist introduces a distributed graph algorithm framework that optimizes communication and memory access, significantly improving performance on large-scale graph processing tasks.
Contribution
It presents a novel analysis-transformation framework and an optimized RMA-based reduction substrate for scalable distributed graph algorithms.
Findings
Outperforms d-Galois by 2.05x in SSSP
Outperforms DRONE by 1.44x in SSSP
Provides scalable distributed graph algorithm implementation
Abstract
Relational data, occurring in the real world, are often structured as graphs, which provide the logical abstraction required to make analytical derivations simpler. As graphs get larger, the irregular access patterns exhibited in most graph algorithms, hamper performance. This, along with NUMA and physical memory limits, results in scaling complexities with sequential/shared memory frameworks. StarPlat's MPI backend abstracts away the programmatic complexity involved in designing optimal distributed graph algorithms. It provides an instrument for coding graph algorithms that scale over distributed memory. In this work, we provide an analysis-transformation framework that leverages general semantics associated with iterations involving nodes and their neighbors, within StarPlat, to aggregate communication. The framework scans for patterns that warrant re-ordering in neighborhood access…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Advanced Graph Neural Networks · Parallel Computing and Optimization Techniques
