Generating Clarifying Questions for Query Refinement in Source Code Search

TL;DR

This paper introduces a method for generating natural clarifying questions to improve source code search, reducing search time and enhancing query refinement by mimicking human question-asking behavior.

Contribution

It proposes a novel approach for automatic clarifying question generation in code search, leveraging function names and comments, and demonstrates its effectiveness through synthetic and human studies.

Findings

Outperformed keyword-based methods in synthetic tests.

Reduced search duration in human studies.

Enhanced query refinement process in code search.

Abstract

In source code search, a common information-seeking strategy involves providing a short initial query with a broad meaning, and then iteratively refining the query using terms gleaned from the results of subsequent searches. This strategy requires programmers to spend time reading search results that are irrelevant to their development needs. In contrast, when programmers seek information from other humans, they typically refine queries by asking and answering clarifying questions. Clarifying questions have been shown to benefit general-purpose search engines, but have not been examined in the context of code search. We present a method for generating natural-sounding clarifying questions using information extracted from function names and comments. Our method outperformed a keyword-based method for single-turn refinement in synthetic studies, and was associated with shorter search duration in human studies.