AI-Mediated Code Comment Improvement

TL;DR

This paper presents an AI-based approach using large language models to automatically improve code comments along various quality axes, with an emphasis on reproducibility and in-house deployment.

Contribution

It introduces a novel procedure leveraging GPT-4o for comment rewriting and distills it into a smaller model for local use, enhancing comment quality in software development.

Findings

The approach improves code comment quality along multiple axes.

Both GPT-4o and distilled models effectively enhance comments.

All data and code are publicly available for reproducibility.

Abstract

This paper describes an approach to improve code comments along different quality axes by rewriting those comments with customized Artificial Intelligence (AI)-based tools. We conduct an empirical study followed by grounded theory qualitative analysis to determine the quality axes to improve. Then we propose a procedure using a Large Language Model (LLM) to rewrite existing code comments along the quality axes. We implement our procedure using GPT-4o, then distil the results into a smaller model capable of being run in-house, so users can maintain data custody. We evaluate both our approach using GPT-4o and the distilled model versions. We show in an evaluation how our procedure improves code comments along the quality axes. We release all data and source code in an online repository for reproducibility.