MARCO: Multi-Agent Code Optimization with Real-Time Knowledge Integration for High-Performance Computing

TL;DR

MARCO is a multi-agent framework that improves high-performance computing code generated by LLMs by integrating real-time knowledge retrieval and iterative optimization, significantly enhancing performance.

Contribution

The paper introduces MARCO, a novel multi-agent system that refines LLM-generated HPC code using real-time knowledge retrieval and feedback loops, addressing limitations of general-purpose models.

Findings

Achieves 14.6% average runtime reduction over baseline LLM.

Web-search component improves performance by 30.9%.

Demonstrates effectiveness of multi-agent systems in HPC code optimization.

Abstract

Large language models (LLMs) have transformed software development through code generation capabilities, yet their effectiveness for high-performance computing (HPC) remains limited. HPC code requires specialized optimizations for parallelism, memory efficiency, and architecture-specific considerations that general-purpose LLMs often overlook. We present MARCO (Multi-Agent Reactive Code Optimizer), a novel framework that enhances LLM-generated code for HPC through a specialized multi-agent architecture. MARCO employs separate agents for code generation and performance evaluation, connected by a feedback loop that progressively refines optimizations. A key innovation is MARCO's web-search component that retrieves real-time optimization techniques from recent conference proceedings and research publications, bridging the knowledge gap in pre-trained LLMs. Our extensive evaluation on the LeetCode 75 problem set demonstrates that MARCO achieves a 14.6\% average runtime reduction compared to Claude 3.5 Sonnet alone, while the integration of the web-search component yields a 30.9\% performance improvement over the base MARCO system. These results highlight the potential of multi-agent systems to address the specialized requirements of high-performance code generation, offering a cost-effective alternative to domain-specific model fine-tuning.