Task-Graph Scheduling Extensions for Efficient Synchronization and Communication
Seonmyeong Bak, Oscar Hernandez, Mark Gates, Piotr Luszczek, Vivek, Sarkar

TL;DR
This paper extends task-graph scheduling to improve synchronization and communication efficiency, combining gang-scheduling and work-stealing, resulting in significant performance gains in linear algebra computations.
Contribution
It introduces a novel task-graph scheduler that supports efficient synchronization and communication, uniquely combining gang-scheduling with work-stealing.
Findings
Achieves up to 36.94% performance improvement in Cholesky factorization.
Demonstrates better overlap of sibling tasks and avoids deadlock.
Outperforms LLVM OMP runtime across multiple linear algebra tasks.
Abstract
Task graphs have been studied for decades as a foundation for scheduling irregular parallel applications and incorporated in programming models such as OpenMP. While many high-performance parallel libraries are based on task graphs, they also have additional scheduling requirements, such as synchronization from inner levels of data parallelism and internal blocking communications. In this paper, we extend task-graph scheduling to support efficient synchronization and communication within tasks. Our scheduler avoids deadlock and oversubscription of worker threads, and refines victim selection to increase the overlap of sibling tasks. To the best of our knowledge, our approach is the first to combine gang-scheduling and work-stealing in a single runtime. Our approach has been evaluated on the SLATE highperformance linear algebra library. Relative to the LLVM OMP runtime, our runtime…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Distributed and Parallel Computing Systems · Advanced Data Storage Technologies
