Demystifying Feature Requests: Leveraging LLMs to Refine Feature Requests in Open-Source Software

TL;DR

This paper introduces a novel LLM-based method to automatically detect and refine ambiguous or incomplete feature requests in open-source software, improving communication and development efficiency.

Contribution

It presents a new automated approach using LLMs to identify and clarify natural language defects in feature requests within OSS projects, addressing a gap in current validation methods.

Findings

The LLM-based method outperforms human annotations in defect detection.

Automated clarification questions improve request quality.

Developers perceive the approach as helpful for understanding requests.

Abstract

The growing popularity and widespread use of software applications (apps) across various domains have driven rapid industry growth. Along with this growth, fast-paced market changes have led to constantly evolving software requirements. Such requirements are often grounded in feature requests and enhancement suggestions, typically provided by users in natural language (NL). However, these requests often suffer from defects such as ambiguity and incompleteness, making them challenging to interpret. Traditional validation methods (e.g., interviews and workshops) help clarify such defects but are impractical in decentralized environments like open-source software (OSS), where change requests originate from diverse users on platforms like GitHub. This paper proposes a novel approach leveraging Large Language Models (LLMs) to detect and refine NL defects in feature requests. Our approach automates the identification of ambiguous and incomplete requests and generates clarification questions (CQs) to enhance their usefulness for developers. To evaluate its effectiveness, we apply our method to real-world OSS feature requests and compare its performance against human annotations. In addition, we conduct interviews with GitHub developers to gain deeper insights into their perceptions of NL defects, the strategies they use to address these defects, and the impact of defects on downstream software engineering (SE) tasks.