OrcaLoca: An LLM Agent Framework for Software Issue Localization

TL;DR

OrcaLoca is a novel LLM agent framework that significantly enhances software issue localization accuracy by integrating advanced search and context techniques, setting new state-of-the-art results in function matching and patch generation.

Contribution

The paper introduces OrcaLoca, a new LLM agent framework that combines priority scheduling, relevance scoring, and distance-aware pruning to improve software issue localization accuracy.

Findings

Achieves 65.33% function match rate on SWE-bench Lite.

Improves final resolved rate by 6.33 percentage points.

Sets new open-source state-of-the-art in software localization.

Abstract

Recent developments in Large Language Model (LLM) agents are revolutionizing Autonomous Software Engineering (ASE), enabling automated coding, problem fixes, and feature improvements. However, localization -- precisely identifying software problems by navigating to relevant code sections -- remains a significant challenge. Current approaches often yield suboptimal results due to a lack of effective integration between LLM agents and precise code search mechanisms. This paper introduces OrcaLoca, an LLM agent framework that improves accuracy for software issue localization by integrating priority-based scheduling for LLM-guided action, action decomposition with relevance scoring, and distance-aware context pruning. Experimental results demonstrate that OrcaLoca becomes the new open-source state-of-the-art (SOTA) in function match rate (65.33%) on SWE-bench Lite. It also improves the final resolved rate of an open-source framework by 6.33 percentage points through its patch generation integration.