Scalable Shared-Memory Hypergraph Partitioning

TL;DR

This paper introduces Mt-KaHyPar, a scalable shared-memory hypergraph partitioner that leverages parallel algorithms to achieve high-quality solutions efficiently on large instances, outperforming existing tools in speed and quality.

Contribution

It presents the first shared-memory multilevel hypergraph partitioner with fully parallel techniques, achieving significant speedups and solution quality improvements over existing methods.

Findings

Achieves up to 51x speedup with 64 cores

Outperforms Zoltan on 95% of instances in quality and speed

Faster than PaToH on 4 cores while producing better solutions

Abstract

Hypergraph partitioning is an important preprocessing step for optimizing data placement and minimizing communication volumes in high-performance computing applications. To cope with ever growing problem sizes, it has become increasingly important to develop fast parallel partitioning algorithms whose solution quality is competitive with existing sequential algorithms. To this end, we present Mt-KaHyPar, the first shared-memory multilevel hypergraph partitioner with parallel implementations of many techniques used by the sequential, high-quality partitioning systems: a parallel coarsening algorithm that uses parallel community detection as guidance, initial partitioning via parallel recursive bipartitioning with work-stealing, a scalable label propagation refinement algorithm, and the first fully-parallel direct $k$-way formulation of the classical FM algorithm. Experiments performed on a large benchmark set of instances from various application domains demonstrate the scalability and effectiveness of our approach. With 64 cores, we observe self-relative speedups of up to 51 and a harmonic mean speedup of 23.5. In terms of solution quality, we outperform the distributed hypergraph partitioner Zoltan on 95% of the instances while also being a factor of 2.1 faster. With just four cores,Mt-KaHyPar is also slightly faster than the fastest sequential multilevel partitioner PaToH while producing better solutions on 83% of all instances. The sequential high-quality partitioner KaHyPar still finds better solutions than our parallel approach, especially when using max-flow-based refinement. This, however, comes at the cost of considerably longer running times.