Implementing Multi-GPU Scientific Computing Miniapps Across Performance Portable Frameworks

TL;DR

This paper compares the performance of four performance portability frameworks—Kokkos, OpenMP, RAJA, and OCCA—in multi-GPU scientific applications, revealing variability and areas for optimization on exascale hardware.

Contribution

It provides preliminary performance insights into how different frameworks perform for scientific applications on multi-GPU systems, highlighting strengths and limitations of each.

Findings

OCCA shows faster execution for small problems due to JIT compilation.

OpenMP performs poorly in inter-node communication scenarios.

Framework performance varies significantly, indicating need for further optimization.

Abstract

Scientific computing in the exascale era demands increased computational power to solve complex problems across various domains. With the rise of heterogeneous computing architectures the need for vendor-agnostic, performance portability frameworks has been highlighted. Libraries like Kokkos have become essential for enabling high-performance computing applications to execute efficiently across different hardware platforms with minimal code changes. In this direction, this paper presents preliminary time-to-solution results for two representative scientific computing applications: an N-body simulation and a structured grid simulation. Both applications used a distributed memory approach and hardware acceleration through four performance portability frameworks: Kokkos, OpenMP, RAJA, and OCCA. Experiments conducted on a single node of the Polaris supercomputer using four NVIDIA A100 GPUs revealed significant performance variability among frameworks. OCCA demonstrated faster execution times for small-scale validation problems, likely due to JIT compilation, however its lack of optimized reduction algorithms may limit scalability for larger simulations while using its out of the box API. OpenMP performed poorly in the structured grid simulation most likely due to inefficiencies in inter-node data synchronization and communication. These findings highlight the need for further optimization to maximize each framework's capabilities. Future work will focus on enhancing reduction algorithms, data communication, memory management, as wells as performing scalability studies, and a comprehensive statistical analysis to evaluate and compare framework performance.