Hardware locality-aware partitioning and dynamic load-balancing of unstructured meshes for large-scale scientific applications

TL;DR

This paper introduces TreePart, an open-source topology-aware hierarchical mesh partitioning and load-balancing tool that adapts to hardware topology for large-scale scientific simulations, improving efficiency and scalability.

Contribution

The paper presents TreePart, a novel framework that automatically detects hardware topology and optimally chooses partitioning strategies for unstructured meshes in HPC applications.

Findings

Successfully integrated into in-house code for large-eddy simulation

Demonstrated improved load-balancing and scalability

Provides scalable abstractions for HPC code optimization

Abstract

We present an open-source topology-aware hierarchical unstructured mesh partitioning and load-balancing tool TreePart. The framework provides powerful abstractions to automatically detect and build hierarchical MPI topology resembling the hardware at runtime. Using this information it intelligently chooses between shared and distributed parallel algorithms for partitioning and load-balancing. It provides a range of partitioning methods by interfacing with existing shared and distributed memory parallel partitioning libraries. It provides powerful and scalable abstractions like one-sided distributed dictionaries and MPI3 shared memory based halo communicators for optimising HPC codes. The tool was successfully integrated into our in-house code and we present results from a large-eddy simulation of a combustion problem.