Performance Measurements within Asynchronous Task-based Runtime Systems: A Double White Dwarf Merger as an Application

TL;DR

This paper investigates performance measurement techniques in asynchronous task-based runtime systems, specifically HPX, using a complex astrophysics application to analyze bottlenecks and energy consumption with minimal overhead.

Contribution

It introduces application-specific performance counters for HPX, enabling integrated physical and performance data visualization in a real-world astrophysics simulation.

Findings

Overhead of performance measurement is approximately 1%.

Performance data helps identify bottlenecks in adaptive grid refinement.

Measurements are feasible on large-scale supercomputers.

Abstract

Analyzing performance within asynchronous many-task-based runtime systems is challenging because millions of tasks are launched concurrently. Especially for long-term runs the amount of data collected becomes overwhelming. We study HPX and its performance-counter framework and APEX to collect performance data and energy consumption. We added HPX application-specific performance counters to the Octo-Tiger full 3D AMR astrophysics application. This enables the combined visualization of physical and performance data to highlight bottlenecks with respect to different solvers. We examine the overhead introduced by these measurements, which is around 1%, with respect to the overall application runtime. We perform a convergence study for four different levels of refinement and analyze the application's performance with respect to adaptive grid refinement. The measurements' overheads are small, enabling the combined use of performance data and physical properties with the goal of improving the code's performance. All of these measurements were obtained on NERSC's Cori, Louisiana Optical Network Infrastructure's QueenBee2, and Indiana University's Big Red 3.