CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System

TL;DR

CompileAgent is an LLM-based framework that automates repo-level compilation by integrating tools for instruction search and error resolution, significantly improving success rates and scalability.

Contribution

This paper introduces the first LLM-based agent system for automated repo-level compilation, integrating tools and flow strategies to enhance efficiency and success.

Findings

Improves compilation success rate by 10% to 71%.

Demonstrates effectiveness of flow-based agent strategy.

Shows scalability of the proposed system.

Abstract

With open-source projects growing in size and complexity, manual compilation becomes tedious and error-prone, highlighting the need for automation to improve efficiency and accuracy. However, the complexity of compilation instruction search and error resolution makes automatic compilation challenging. Inspired by the success of LLM-based agents in various fields, we propose CompileAgent, the first LLM-based agent framework dedicated to repo-level compilation. CompileAgent integrates five tools and a flow-based agent strategy, enabling interaction with software artifacts for compilation instruction search and error resolution. To measure the effectiveness of our method, we design a public repo-level benchmark CompileAgentBench, and we also design two baselines for comparison by combining two compilation-friendly schemes. The performance on this benchmark shows that our method significantly improves the compilation success rate, ranging from 10% to 71%. Meanwhile, we evaluate the performance of CompileAgent under different agent strategies and verify the effectiveness of the flow-based strategy. Additionally, we emphasize the scalability of CompileAgent, further expanding its application prospects.