OMPILOT: Harnessing Transformer Models for Auto Parallelization to Shared Memory Computing Paradigms

TL;DR

This paper introduces OMPILOT, a transformer-based model for translating C++ code into OpenMP parallel constructs, improving shared-memory parallelization with specialized training and a new evaluation metric.

Contribution

The paper presents a domain-specific transformer model with novel training objectives and a new metric, OMPBLEU, for better translation of parallel code constructs.

Findings

OMPILOT outperforms baseline models in code translation accuracy.

OMPBLEU effectively evaluates the correctness of OpenMP constructs.

Function-level translation captures broader semantic context.

Abstract

Recent advances in large language models (LLMs) have significantly accelerated progress in code translation, enabling more accurate and efficient transformation across programming languages. While originally developed for natural language processing, LLMs have shown strong capabilities in modeling programming language syntax and semantics, outperforming traditional rule-based systems in both accuracy and flexibility. These models have streamlined cross-language conversion, reduced development overhead, and accelerated legacy code migration. In this paper, we introduce OMPILOT, a novel domain-specific encoder-decoder transformer tailored for translating C++ code into OpenMP, enabling effective shared-memory parallelization. OMPILOT leverages custom pre-training objectives that incorporate the semantics of parallel constructs and combines both unsupervised and supervised learning strategies to improve code translation robustness. Unlike previous work that focused primarily on loop-level transformations, OMPILOT operates at the function level to capture a wider semantic context. To evaluate our approach, we propose OMPBLEU, a novel composite metric specifically crafted to assess the correctness and quality of OpenMP parallel constructs, addressing limitations in conventional translation metrics.