OMPar: Automatic Parallelization with AI-Driven Source-to-Source Compilation

TL;DR

OMPar is an AI-driven tool that automates C/C++ code parallelization using Large Language Models to identify parallelizable loops and generate OpenMP pragmas, outperforming traditional methods.

Contribution

Introduces OMPar, a novel AI-based system combining LLMs for assessing loop parallelization and generating OpenMP pragmas, improving accuracy and scalability over existing tools.

Findings

OMPar outperforms traditional parallelization tools in accuracy.

The system effectively handles partial or incomplete codebases.

OMPar demonstrates significant performance gains on benchmark tests.

Abstract

Manual parallelization of code remains a significant challenge due to the complexities of modern software systems and the widespread adoption of multi-core architectures. This paper introduces OMPar, an AI-driven tool designed to automate the parallelization of C/C++ code using OpenMP pragmas. OMPar integrates Large Language Models (LLMs) through two key components: OMPify, which assesses loop parallelization potential, and MonoCoder-OMP, a new fine-tuned model which generates precise OpenMP pragmas. The evaluation of OMPar follows the same rigorous process applied to traditional tools like source-to-source AutoPar and ICPC compilers: (1) ensuring the generated code compiles and runs correctly in serial form, (2) assessing performance with the gradual addition of threads and corresponding physical cores, and (3) verifying and validating the correctness of the code's output. Benchmarks from HeCBench and ParEval are used to evaluate accuracy and performance. Experimental results demonstrate that OMPar significantly outperforms traditional methods, achieving higher accuracy in identifying parallelizable loops and generating efficient pragmas. Beyond accuracy, OMPar offers advantages such as the ability to work on partial or incomplete codebases and the capacity to continuously learn from new code patterns, enhancing its parallelization capabilities over time. These results underscore the potential of LLMs in revolutionizing automatic parallelization techniques, paving the way for more efficient and scalable parallel computing systems.