Generating Bindings in MPICH

TL;DR

This paper presents a Python-based code generation toolbox that automates the creation of MPI language bindings in MPICH, significantly reducing manual coding effort and minimizing bugs.

Contribution

It introduces a Python scripting tool that automates MPI binding generation, replacing extensive manual code and facilitating API extensions and profiling interface prototyping.

Findings

Replaces 70,000 lines of manual code with 5,000 lines of Python scripts.

Eliminates code duplication and reduces bugs in MPI bindings.

Enables easier API extension and new profiling interface development.

Abstract

The MPI Forum has recently adopted a Python scripting engine for generating the API text in the standard document. As a by-product, it made available reliable and rich descriptions of all MPI functions that are suited for scripting tools. Using these extracted API information, we developed a Python code generation toolbox to generate the language binding layers in MPICH. The toolbox replaces nearly 70,000 lines of manually maintained C and Fortran 2008 binding code with around 5,000 lines of Python scripts plus some simple configuration. In addition to completely eliminating code duplication in the binding layer and avoiding bugs from manual code copying , the code generation also minimizes the effort for API extension and code instrumentation. This is demonstrated in our implementation of MPI-4 large count functions and the prototyping of a next generation MPI profiling interface, QMPI.