ParaCodex: A Profiling-Guided Autonomous Coding Agent for Reliable Parallel Code Generation and Translation

TL;DR

ParaCodex is an autonomous coding system that improves parallel GPU code generation and translation, achieving significant speedups and high success rates through profiling-guided refinement and domain-specific techniques.

Contribution

It introduces ParaCodex, a novel autonomous system that enhances OpenMP GPU offload code translation and generation with profiling and correctness checks, outperforming baseline methods.

Findings

Succeeded on all 31 valid kernels after filtering out failures.

Achieved geometric-mean speedups of 3x on HeCBench and 5x on Rodinia.

Maintained high compilation and validation rates in CUDA to OpenMP translation.

Abstract

Parallel programming is central to HPC and AI, but producing code that is correct and fast remains challenging, especially for OpenMP GPU offload, where data movement and tuning dominate. Autonomous coding agents can compile, test, and profile on target hardware, but outputs are brittle without domain scaffolding. We present ParaCodex, an HPC-engineer workflow that turns a Codex-based agent into an autonomous OpenMP GPU offload system using staged hotspot analysis, explicit data planning, correctness gating, and profiling-guided refinement. We evaluate translation from serial CPU kernels to OpenMP GPU offload kernels on HeCBench, Rodinia, and NAS. After excluding five kernels, ParaCodex succeeded on all 31 valid kernels. The generated kernels improved GPU time over reference OpenMP implementations in 25/31 cases, achieving geometric-mean speedups of 3x on HeCBench and 5x on Rodinia, and outperforming a zero-shot Codex baseline on all suites. We also evaluate CUDA to OpenMP offload translation on ParEval, where ParaCodex maintains high compilation and validation rates in code-only and end-to-end settings.