An Empirical Study of Refactoring Engine Bugs

TL;DR

This study systematically analyzes bugs in refactoring engines across popular IDEs, revealing common issues, root causes, and providing guidelines to improve bug detection and debugging in automated code refactoring tools.

Contribution

First comprehensive analysis of refactoring engine bugs, identifying their types, causes, symptoms, and providing guidelines for future bug detection and fixing.

Findings

Analyzed bugs in Eclipse, IntelliJ IDEA, and Netbeans refactoring engines.

Identified 12 key findings about bug types, causes, and symptoms.

Discovered 130 new bugs in latest engine versions.

Abstract

Refactoring is a critical process in software development, aiming at improving the internal structure of code while preserving its external behavior. Refactoring engines are integral components of modern Integrated Development Environments (IDEs) and can automate or semi-automate this process to enhance code readability, reduce complexity, and improve the maintainability of software products. Like traditional software systems, refactoring engines can generate incorrect refactored programs, resulting in unexpected behaviors or even crashes. In this paper, we present the first systematic study of refactoring engine bugs by analyzing bugs arising in three popular refactoring engines (i.e., Eclipse, IntelliJ IDEA, and Netbeans). We analyzed these bugs according to their refactoring types, symptoms, root causes, and triggering conditions. We obtained 12 findings and provided a series of valuable guidelines for future work on refactoring bug detection and debugging. Furthermore, our transferability study revealed 130 new bugs in the latest version of those refactoring engines. Among the 21 bugs we submitted, 10 bugs are confirmed by their developers, and seven of them have already been fixed.