Toward Linking Declined Proposals and Source Code: An Exploratory Study on the Go Repository

TL;DR

This study explores linking declined proposals in the Go repository to source code using an LLM-driven approach, revealing factors affecting link accuracy and challenges in handling redundant or vague discussions.

Contribution

It introduces the first method to establish traceability links between declined contributions and source code, utilizing an LLM-driven pipeline for the Go repository.

Findings

Achieved 83.6% accuracy in selecting correct granularity.

Generated correct links with a mean precision of 64.3%.

Identified challenges due to redundant discussions and lack of concrete details.

Abstract

Traceability links are key information sources for software developers, connecting software artifacts. Such links play an important role, particularly between contribution artifacts and their corresponding source code. Through these links, developers can trace the discussions in contributions and uncover design rationales, constraints, and security concerns. Previous studies have mainly examined accepted contributions, while those declined after discussion have been overlooked. Declined-contribution discussions capture valuable design rationale and implicit decision criteria, revealing why features are accepted or rejected. Our prior work also shows developers often revisit and resubmit declined contributions, making traceability to them useful. In this study, we present the first attempt to establish traceability links between declined contributions and related source code. We propose a linking approach and conduct an empirical analysis of the generated links to discuss the factors that affect link generation. As our dataset, we use proposals from the official Go repository, which are GitHub issues used to propose new features or language changes. To link declined proposals to source code, we design an LLM-driven pipeline. Our results show that the pipeline selected the correct granularity for each declined proposal with an accuracy of 0.836, and generated correct links at that granularity with a mean precision of 0.643. To clarify the challenges of linking declined proposals, we conduct a failure analysis of instances where the pipeline failed to generate links. In these cases, discussions were often redundant and lacked concrete information (e.g., details on how the feature should be implemented).