RefAgent: A Multi-agent LLM-based Framework for Automatic Software Refactoring

TL;DR

RefAgent introduces a multi-agent LLM framework that autonomously performs software refactoring, significantly improving code quality, refactoring accuracy, and alignment with developer practices across multiple open-source Java projects.

Contribution

This paper presents the novel RefAgent framework, a multi-agent LLM-based system that automates software refactoring with dynamic decision-making, self-reflection, and tool integration, outperforming single-agent and traditional methods.

Findings

RefAgent achieves a median 90% unit test pass rate.

Reduces code smells by a median of 52.5%.

Attains a median F1-score of 79.15% in identifying refactoring opportunities.

Abstract

Large Language Models (LLMs) have substantially influenced various software engineering tasks. Indeed, in the case of software refactoring, traditional LLMs have shown the ability to reduce development time and enhance code quality. However, these LLMs often rely on static, detailed instructions for specific tasks. In contrast, LLM-based agents can dynamically adapt to evolving contexts and autonomously make decisions by interacting with software tools and executing workflows. In this paper, we explore the potential of LLM-based agents in supporting refactoring activities. Specifically, we introduce RefAgent, a multi-agent LLM-based framework for end-to-end software refactoring. RefAgent consists of specialized agents responsible for planning, executing, testing, and iteratively refining refactorings using self-reflection and tool-calling capabilities. We evaluate RefAgent on eight open-source Java projects, comparing its effectiveness against a single-agent approach, a search-based refactoring tool, and historical developer refactorings. Our assessment focuses on: (1) the impact of generated refactorings on software quality, (2) the ability to identify refactoring opportunities, and (3) the contribution of each LLM agent through an ablation study. Our results show that RefAgent achieves a median unit test pass rate of 90%, reduces code smells by a median of 52.5%, and improves key quality attributes (e.g., reusability) by a median of 8.6%. Additionally, it closely aligns with developer refactorings and the search-based tool in identifying refactoring opportunities, attaining a median F1-score of 79.15% and 72.7%, respectively. Compared to single-agent approaches, RefAgent improves the median unit test pass rate by 64.7% and the median compilation success rate by 40.1%. These findings highlight the promise of multi-agent architectures in advancing automated software refactoring.