Investigating the Impact of Code Comment Inconsistency on Bug Introducing
Shiva Radmanesh, Aaron Imani, Iftekhar Ahmed, Mohammad Moshirpour

TL;DR
This study uses GPT-3.5 to analyze how outdated code comments contribute to bugs, revealing that inconsistencies significantly increase bug risk, especially immediately after they occur.
Contribution
It demonstrates GPT-3.5's effectiveness in detecting code-comment inconsistencies and provides a temporal analysis of their impact on bug introduction.
Findings
Inconsistent comments are 1.5 times more likely to cause bugs.
The impact of comment inconsistency is highest immediately after changes.
GPT-3.5 outperforms other methods in inconsistency detection.
Abstract
Code comments are essential for clarifying code functionality, improving readability, and facilitating collaboration among developers. Despite their importance, comments often become outdated, leading to inconsistencies with the corresponding code. This can mislead developers and potentially introduce bugs. Our research investigates the impact of code-comment inconsistency on bug introduction using large language models, specifically GPT-3.5. We first compare the performance of the GPT-3.5 model with other state-of-the-art methods in detecting these inconsistencies, demonstrating the superiority of GPT-3.5 in this domain. Additionally, we analyze the temporal evolution of code-comment inconsistencies and their effect on bug proneness over various timeframes using GPT-3.5 and Odds ratio analysis. Our findings reveal that inconsistent changes are around 1.5 times more likely to lead to a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Natural Language Processing Techniques · Software Engineering Techniques and Practices
