HPCAgentTester: A Multi-Agent LLM Approach for Enhanced HPC Unit Test Generation

TL;DR

HPCAgentTester is a multi-agent LLM framework that automates and improves unit test generation for HPC applications, effectively addressing parallelism and synchronization challenges.

Contribution

It introduces a collaborative multi-agent LLM approach for generating context-aware HPC unit tests, enhancing bug detection and test correctness.

Findings

Significantly improves test compilation rates.

Effectively detects subtle parallel bugs.

Outperforms standalone LLMs in test generation.

Abstract

Unit testing in High-Performance Computing (HPC) is critical but challenged by parallelism, complex algorithms, and diverse hardware. Traditional methods often fail to address non-deterministic behavior and synchronization issues in HPC applications. This paper introduces HPCAgentTester, a novel multi-agent Large Language Model (LLM) framework designed to automate and enhance unit test generation for HPC software utilizing OpenMP and MPI. HPCAgentTester employs a unique collaborative workflow where specialized LLM agents (Recipe Agent and Test Agent) iteratively generate and refine test cases through a critique loop. This architecture enables the generation of context-aware unit tests that specifically target parallel execution constructs, complex communication patterns, and hierarchical parallelism. We demonstrate HPCAgentTester's ability to produce compilable and functionally correct tests for OpenMP and MPI primitives, effectively identifying subtle bugs that are often missed by conventional techniques. Our evaluation shows that HPCAgentTester significantly improves test compilation rates and correctness compared to standalone LLMs, offering a more robust and scalable solution for ensuring the reliability of parallel software systems.