Prompt-Driven Code Summarization: A Systematic Literature Review

TL;DR

This paper systematically reviews how prompt engineering techniques influence the effectiveness of large language models in automatic code summarization and software documentation tasks.

Contribution

It categorizes prompting strategies, evaluates their effectiveness, and highlights research gaps to guide future work in LLM-based code documentation.

Findings

Prompt engineering significantly impacts LLM performance in code summarization.

Few-shot and chain-of-thought prompting show promising results.

Evaluation practices vary, affecting comparability of research outcomes.

Abstract

Software documentation is essential for program comprehension, developer onboarding, code review, and long-term maintenance. Yet producing quality documentation manually is time-consuming and frequently yields incomplete or inconsistent results. Large language models (LLMs) offer a promising solution by automatically generating natural language descriptions from source code, helping developers understand code more efficiently, facilitating maintenance, and supporting downstream activities such as defect localization and commit message generation. However, the effectiveness of LLMs in documentation tasks critically depends on how they are prompted. Properly structured instructions can substantially improve model performance, making prompt engineering-the design of input prompts to guide model behavior-a foundational technique in LLM-based software engineering. Approaches such as few-shot prompting, chain-of-thought reasoning, retrieval-augmented generation, and zero-shot learning show promise for code summarization, yet current research remains fragmented. There is limited understanding of which prompting strategies work best, for which models, and under what conditions. Moreover, evaluation practices vary widely, with most studies relying on overlap-based metrics that may not capture semantic quality. This systematic literature review consolidates existing evidence, categorizes prompting paradigms, examines their effectiveness, and identifies gaps to guide future research and practical adoption.