AUTOGENICS: Automated Generation of Context-Aware Inline Comments for Code Snippets on Programming Q&A Sites Using LLM

TL;DR

AUTOGENICS is a tool that leverages large language models to automatically generate context-aware inline comments for code snippets on Stack Overflow, improving readability and understanding for developers.

Contribution

The paper introduces AUTOGENICS, a novel system that enhances LLM-generated inline comments by incorporating question context and noise reduction, addressing limitations of previous comment generation methods.

Findings

LLMs can generate effective inline comments for SO code snippets.

AUTOGENICS outperforms standard LLMs in comment quality metrics.

User surveys confirm perceived usefulness of AUTOGENICS comments.

Abstract

Inline comments in the source code facilitate easy comprehension, reusability, and enhanced readability. However, code snippets in answers on Q&A sites like Stack Overflow (SO) often lack comments because answerers volunteer their time and often skip comments or explanations due to time constraints. Existing studies show that these online code examples are difficult to read and understand, making it difficult for developers (especially novices) to use them correctly and leading to misuse. Given these challenges, we introduced AUTOGENICS, a tool designed to integrate with SO to generate effective inline comments for code snippets in SO answers exploiting large language models (LLMs). Our contributions are threefold. First, we randomly select 400 answer code snippets from SO and generate inline comments for them using LLMs. We then manually evaluate these comments' effectiveness using four key metrics: accuracy, adequacy, conciseness, and usefulness. Overall, LLMs demonstrate promising effectiveness in generating inline comments for SO answer code snippets. Second, we surveyed 14 active SO users to perceive the effectiveness of these inline comments. The survey results are consistent with our previous manual evaluation. However, according to our evaluation, LLMs-generated comments are less effective for shorter code snippets and sometimes produce noisy comments. Third, to address the gaps, we introduced AUTOGENICS, which extracts additional context from question texts and generates context-aware inline comments. It also optimizes comments by removing noise (e.g., comments in import statements and variable declarations). We evaluate the effectiveness of AUTOGENICS-generated comments using the same four metrics that outperform those of standard LLMs. AUTOGENICS might (a) enhance code comprehension, (b) save time, and improve developers' ability to learn and reuse code more accurately.