LLMCup: Ranking-Enhanced Comment Updating with LLMs
Hua Ge, Juan Zhai, Minxue Pan, Fusen He, Ziyue Tan

TL;DR
This paper introduces LLMCup, a framework leveraging multiple prompt strategies and ranking models with large language models to improve automatic comment updating in software projects, significantly outperforming previous methods.
Contribution
The paper proposes a novel comment updating framework that combines diverse prompt strategies and a ranking model to enhance LLM-based comment updates, addressing previous limitations.
Findings
LLMCup outperforms state-of-the-art baselines in accuracy and BLEU-4 metrics.
Experimental results show significant improvements over previous methods.
User studies indicate LLMCup can sometimes surpass human-written comment updates.
Abstract
While comments are essential for enhancing code readability and maintainability in modern software projects, developers are often motivated to update code but not comments, leading to outdated or inconsistent documentation that hinders future understanding and maintenance. Recent approaches such as CUP and HebCup have attempted automatic comment updating using neural sequence-to-sequence models and heuristic rules, respectively. However, these methods can miss or misinterpret crucial information during comment updating, resulting in inaccurate comments, and they often struggle with complex update scenarios. Given these challenges, a promising direction lies in leveraging large language models (LLMs), which have shown impressive performance in software engineering tasks such as comment generation, code synthesis, and program repair. This suggests their strong potential to capture the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
