P4OMP: Retrieval-Augmented Prompting for OpenMP Parallelism in Serial Code

TL;DR

P4OMP is a retrieval-augmented framework that uses large language models and structured knowledge to reliably transform serial C/C++ code into correct, parallel OpenMP code without fine-tuning, achieving perfect compilation success on a diverse benchmark.

Contribution

This work introduces P4OMP, the first retrieval-augmented prompting system for OpenMP parallelization that ensures correctness without model fine-tuning or compiler modifications.

Findings

Achieves 100% compilation success on 108 real-world programs.

Outperforms baseline GPT-3.5-Turbo without retrieval.

Demonstrates strong runtime scaling on HPC benchmarks.

Abstract

We present P4OMP, a retrieval-augmented framework for transforming serial C/C++ code into OpenMP-annotated parallel code using large language models (LLMs). To our knowledge, this is the first system to apply retrieval-based prompting for OpenMP pragma correctness without model fine-tuning or compiler instrumentation. P4OMP leverages Retrieval-Augmented Generation (RAG) with structured instructional knowledge from OpenMP tutorials to improve the reliability of prompt-driven code generation. By grounding generation in the retrieved context, P4OMP improves syntactic correctness compared to baseline prompting with GPT-3.5-Turbo. We evaluate P4OMP against a baseline, GPT-3.5-Turbo without retrieval, on a comprehensive benchmark of 108 real-world C++ programs drawn from Stack Overflow, PolyBench, and NAS benchmark suites. P4OMP achieves 100% compilation success on all parallelizable cases, while the baseline fails to compile in 20 out of 108 cases. Six cases that rely on non-random-access iterators or thread-unsafe constructs are excluded due to fundamental OpenMP limitations. A detailed analysis demonstrates how P4OMP consistently avoids scoping errors, syntactic misuse, and invalid directive combinations that commonly affect baseline-generated code. We further demonstrate strong runtime scaling across seven compute-intensive benchmarks on an HPC cluster. P4OMP offers a robust, modular pipeline that significantly improves the reliability and applicability of LLM-generated OpenMP code.