AutoParLLM: GNN-guided Context Generation for Zero-Shot Code Parallelization using LLMs

TL;DR

AutoParLLM leverages GNN-guided context generation to enhance zero-shot parallel code generation with LLMs, significantly improving code quality and speedup on benchmark suites.

Contribution

The paper introduces AutoParLLM, a novel GNN-guided context generation method that improves zero-shot parallel code generation with LLMs, and proposes a new evaluation metric for parallel code quality.

Findings

AutoParLLM improves NAS benchmark scores by 19.9%.

AutoParLLM enhances GPT-4's speedup by approximately 17%.

The proposed heirScore{} effectively evaluates parallel code quality.

Abstract

In-Context Learning (ICL) has been shown to be a powerful technique to augment the capabilities of LLMs for a diverse range of tasks. This work proposes \ourtool, a novel way to generate context using guidance from graph neural networks (GNNs) to generate efficient parallel codes. We evaluate \ourtool \xspace{} on $12$ applications from two well-known benchmark suites of parallel codes: NAS Parallel Benchmark and Rodinia Benchmark. Our results show that \ourtool \xspace{} improves the state-of-the-art LLMs (e.g., GPT-4) by 19.9\% in NAS and 6.48\% in Rodinia benchmark in terms of CodeBERTScore for the task of parallel code generation. Moreover, \ourtool \xspace{} improves the ability of the most powerful LLM to date, GPT-4, by achieving $\approx$17\% (on NAS benchmark) and $\approx$16\% (on Rodinia benchmark) better speedup. In addition, we propose \ourscore \xspace{} for evaluating the quality of the parallel code and show its effectiveness in evaluating parallel codes. \ourtool \xspace is available at https://github.com/quazirafi/AutoParLLM.git.