Bugs in the Shadows: Static Detection of Faulty Python Refactorings

TL;DR

This paper presents a static analysis method to detect type errors caused by automated refactoring in Python, uncovering bugs in existing tools and IDEs to improve software reliability.

Contribution

The work introduces a novel static analysis technique specifically designed for identifying type errors in Python refactorings, validated on real-world projects and tools.

Findings

Detected 29 bugs across four refactoring types

Uncovered issues in popular IDEs like PyCharm and PyDev

All bugs were reported and some acknowledged by developers

Abstract

Python is a widely adopted programming language, valued for its simplicity and flexibility. However, its dynamic type system poses significant challenges for automated refactoring - an essential practice in software evolution aimed at improving internal code structure without changing external behavior. Understanding how type errors are introduced during refactoring is crucial, as such errors can compromise software reliability and reduce developer productivity. In this work, we propose a static analysis technique to detect type errors introduced by refactoring implementations for Python. We evaluated our technique on Rope refactoring implementations, applying them to open-source Python projects. Our analysis uncovered 29 bugs across four refactoring types from a total of 1,152 refactoring attempts. Several of these issues were also found in widely used IDEs such as PyCharm and PyDev. All reported bugs were submitted to the respective developers, and some of them were acknowledged and accepted. These results highlight the need to improve the robustness of current Python refactoring tools to ensure the correctness of automated code transformations and support reliable software maintenance.