Copilot-in-the-Loop: Fixing Code Smells in Copilot-Generated Python Code using Copilot
Beiqi Zhang, Peng Liang, Qiong Feng, Yujia Fu, Zengyang Li

TL;DR
This study investigates the prevalence of code smells in Python code generated by GitHub Copilot and evaluates how effectively Copilot Chat can detect and fix these issues using different prompts.
Contribution
The paper introduces a dataset of 102 code smells in Copilot-generated Python code and assesses Copilot Chat's ability to detect and repair these smells, highlighting prompt influence.
Findings
8 out of 10 code smell types detected in Copilot-generated code
Copilot Chat fixes up to 87.1% of detected code smells
More detailed prompts improve fixing effectiveness
Abstract
As one of the most popular dynamic languages, Python experiences a decrease in readability and maintainability when code smells are present. Recent advancements in Large Language Models have sparked growing interest in AI-enabled tools for both code generation and refactoring. GitHub Copilot is one such tool that has gained widespread usage. Copilot Chat, released in September 2023, functions as an interactive tool aimed at facilitating natural language-powered coding. However, limited attention has been given to understanding code smells in Copilot-generated Python code and Copilot Chat's ability to fix the code smells. To this end, we built a dataset comprising 102 code smells in Copilot-generated Python code. Our aim is to first explore the occurrence of code smells in Copilot-generated Python code and then evaluate the effectiveness of Copilot Chat in fixing these code smells…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Software System Performance and Reliability · Scientific Computing and Data Management
