OMB-Py: Python Micro-Benchmarks for Evaluating Performance of MPI Libraries on HPC Systems

TL;DR

This paper introduces OMB-Py, the first benchmark suite for evaluating MPI communication performance in Python HPC applications, covering popular libraries and revealing minimal overhead of mpi4py.

Contribution

The paper presents OMB-Py, a novel Python micro-benchmark suite for MPI communication, filling a gap in performance evaluation tools for Python HPC applications.

Findings

mpi4py has minimal overhead compared to native MPI

OMB-Py covers multiple Python libraries like NumPy and CuPy

First benchmark suite for Python MPI communication performance

Abstract

Python has become a dominant programming language for emerging areas like Machine Learning (ML), Deep Learning (DL), and Data Science (DS). An attractive feature of Python is that it provides easy-to-use programming interface while allowing library developers to enhance performance of their applications by harnessing the computing power offered by High Performance Computing (HPC) platforms. Efficient communication is key to scaling applications on parallel systems, which is typically enabled by the Message Passing Interface (MPI) standard and compliant libraries on HPC hardware. mpi4py is a Python-based communication library that provides an MPI-like interface for Python applications allowing application developers to utilize parallel processing elements including GPUs. However, there is currently no benchmark suite to evaluate communication performance of mpi4py -- and Python MPI codes in general -- on modern HPC systems. In order to bridge this gap, we propose OMB-Py -- Python extensions to the open-source OSU Micro-Benchmark (OMB) suite -- aimed to evaluate communication performance of MPI-based parallel applications in Python. To the best of our knowledge, OMB-Py is the first communication benchmark suite for parallel Python applications. OMB-Py consists of a variety of point-to-point and collective communication benchmark tests that are implemented for a range of popular Python libraries including NumPy, CuPy, Numba, and PyCUDA. Our evaluation reveals that mpi4py introduces a small overhead when compared to native MPI libraries. We plan to publicly release OMB-Py to benefit the Python HPC community.