Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks

TL;DR

This paper introduces CLARITI, a reinforcement learning-based clarification module for software engineering tasks, which effectively identifies valuable questions by analyzing information relevance and answerability, reducing unnecessary queries.

Contribution

It presents a novel reward-driven approach grounded in empirical analysis to improve clarification efficiency in software engineering tasks.

Findings

CLARITI matches GPT-5's resolution rate on underspecified issues.

It generates 41% fewer questions than baseline methods.

Grounding rewards in empirical analysis enhances clarification effectiveness.

Abstract

Humans often specify tasks incompletely, so assistants must know when and how to ask clarifying questions. However, effective clarification remains challenging in software engineering tasks as not all missing information is equally valuable, and questions must target information users can realistically provide. We study clarification in real software engineering tasks by quantifying which types of information most affect task success and which questions elicit useful responses from simulated users. Using Shapley attribution and distributional comparisons, we identify two key properties of effective clarification: task relevance (which information predicts success) and user answerability (what users can realistically provide). We operationalize these properties as multi-stage reinforcement learning rewards to train CLARITI, an 8B-parameter clarification module, that matches GPT-5's resolution rate on underspecified issues while generating 41% fewer questions. Our results suggest that grounding reward design in empirical analysis of information impact and user answerability improves clarification efficiency.