Hybrid-parallel sparse matrix-vector multiplication with explicit communication overlap on current multicore-based systems
Gerald Schubert, Holger Fehske, Georg Hager, Gerhard Wellein

TL;DR
This paper investigates optimized sparse matrix-vector multiplication on multicore clusters, demonstrating that explicit communication overlap with dedicated threads and hybrid MPI/OpenMP strategies improve performance.
Contribution
It introduces a method for explicit communication and computation overlap using dedicated threads, enhancing parallel sparse matrix-vector multiplication performance.
Findings
Explicit overlap with dedicated communication threads improves performance.
Hybrid MPI/OpenMP strategies offer better load balancing.
Performance gains are demonstrated on multicore and Cray XE6 systems.
Abstract
We evaluate optimized parallel sparse matrix-vector operations for several representative application areas on widespread multicore-based cluster configurations. First the single-socket baseline performance is analyzed and modeled with respect to basic architectural properties of standard multicore chips. Beyond the single node, the performance of parallel sparse matrix-vector operations is often limited by communication overhead. Starting from the observation that nonblocking MPI is not able to hide communication cost using standard MPI implementations, we demonstrate that explicit overlap of communication and computation can be achieved by using a dedicated communication thread, which may run on a virtual core. Moreover we identify performance benefits of hybrid MPI/OpenMP programming due to improved load balancing even without explicit communication overlap. We compare performance…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
