EM-Assist: Safe Automated ExtractMethod Refactoring with LLMs

TL;DR

EM-Assist is an IntelliJ plugin that leverages large language models to generate, validate, and rank extract method refactoring suggestions, achieving higher recall and positive developer feedback compared to existing static analysis tools.

Contribution

This paper introduces EM-Assist, the first LLM-based tool for recommending extract method refactorings that align with real-world developer practices.

Findings

EM-Assist achieved a 53.4% recall rate among top-5 recommendations.

It outperformed the previous best static analysis tool with 39.4% recall.

94.4% of industrial developers rated EM-Assist positively.

Abstract

Excessively long methods, loaded with multiple responsibilities, are challenging to understand, debug, reuse, and maintain. The solution lies in the widely recognized Extract Method refactoring. While the application of this refactoring is supported in modern IDEs, recommending which code fragments to extract has been the topic of many research tools. However, they often struggle to replicate real-world developer practices, resulting in recommendations that do not align with what a human developer would do in real life. To address this issue, we introduce EM-Assist, an IntelliJ IDEA plugin that uses LLMs to generate refactoring suggestions and subsequently validates, enhances, and ranks them. Finally, EM-Assist uses the IntelliJ IDE to apply the user-selected recommendation. In our extensive evaluation of 1,752 real-world refactorings that actually took place in open-source projects, EM-Assist's recall rate was 53.4% among its top-5 recommendations, compared to 39.4% for the previous best-in-class tool that relies solely on static analysis. Moreover, we conducted a usability survey with 18 industrial developers and 94.4% gave a positive rating.