Beyond Local Code Optimization: Multi-Agent Reasoning for Software System Optimization

TL;DR

This paper proposes a multi-agent system for optimizing entire software systems, especially microservices, by reasoning about architecture and interactions, leading to significant performance improvements.

Contribution

It introduces a novel multi-agent framework that integrates system-level reasoning for software optimization beyond local code transformations.

Findings

Achieved 36.58% throughput improvement

Reduced response time by 27.81%

Demonstrated effectiveness on microservice systems

Abstract

Large language models and AI agents have recently shown promise in automating software performance optimization, but existing approaches predominantly rely on local, syntax-driven code transformations. This limits their ability to reason about program behavior and capture whole system performance interactions. As modern software increasingly comprises interacting components - such as microservices, databases, and shared infrastructure - effective code optimization requires reasoning about program structure and system architecture beyond individual functions or files. This paper explores the feasibility of whole system optimization for microservices. We introduce a multi-agent framework that integrates control-flow and data-flow representations with architectural and cross-component dependency signals to support system-level performance reasoning. The proposed system is decomposed into coordinated agent roles - summarization, analysis, optimization, and verification - that collaboratively identify cross-cutting bottlenecks and construct multi-step optimization strategies spanning the software stack. We present a proof-of-concept on a microservice-based system that illustrates the effectiveness of our proposed framework, achieving a 36.58% improvement in throughput and a 27.81% reduction in average response time.