Precision in Practice: Knowledge Guided Code Summarizing Grounded in Industrial Expectations

TL;DR

This paper introduces ExpSum, a knowledge-guided code summarization method tailored for industrial needs, which significantly improves summary relevance and developer satisfaction by incorporating domain-specific expectations and constraints.

Contribution

The paper presents ExpSum, a novel expectation-aware code summarization approach that integrates domain knowledge and constraints to produce more aligned and useful summaries for industrial applications.

Findings

ExpSum outperforms baselines with up to 26.71% BLEU-4 improvement.

ExpSum achieves up to 20.10% ROUGE-L improvement.

Summaries generated by ExpSum better match developer expectations.

Abstract

Code summaries are essential for helping developers understand code functionality and reducing maintenance and collaboration costs. Although recent advances in large language models (LLMs) have significantly improved automatic code summarization, the practical usefulness of generated summaries in industrial settings remains insufficiently explored. In collaboration with documentation experts from the industrial HarmonyOS project, we conducted a questionnaire study showing that over 57.4% of code summaries produced by state-of-the-art approaches were rejected due to violations of developers' expectations for industrial documentation. Beyond semantic similarity to reference summaries, developers emphasize additional requirements, including the use of appropriate domain terminology, explicit function categorization, and the avoidance of redundant implementation details. To address these expectations, we propose ExpSum, an expectation-aware code summarization approach that integrates function metadata abstraction, informative metadata filtering, context-aware domain knowledge retrieval, and constraint-driven prompting to guide LLMs in generating structured, expectation-aligned summaries. We evaluate ExpSum on the HarmonyOS project and widely used code summarization benchmarks. Experimental results show that ExpSum consistently outperforms all baselines, achieving improvements of up to 26.71% in BLEU-4 and 20.10% in ROUGE-L on HarmonyOS. Furthermore, LLM-based evaluations indicate that ExpSum-generated summaries better align with developer expectations across other projects, demonstrating its effectiveness for industrial code documentation.