Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions
Harsh Darji, Thibaud Lutellier

TL;DR
This paper presents a novel LLM-based coding assistant that improves code accuracy by asking clarification questions for ambiguous prompts, mimicking human review, and outperforming baseline models in usefulness and user satisfaction.
Contribution
The work introduces a system combining a query classifier and a fine-tuned LLM to generate clarification questions, enhancing coding assistance for ambiguous developer queries.
Findings
Fine-tuned LLM generates more useful clarification questions than zero-shot prompting.
Users prefer clarification questions from the proposed system over baseline models.
The system improves code accuracy and user satisfaction in coding assistance.
Abstract
Large Language Models (LLMs) are increasingly used as coding assistants. However, the ambiguity of the developer's prompt often leads to incorrect code generation, as current models struggle to infer user intent without extensive prompt engineering or external context. This work aims to build an LLM-based coding assistant that mimics the human code review process by asking clarification questions when faced with ambiguous or under-specified queries. Our end-to-end system includes (1) a query classifier trained to detect unclear programming-related queries and (2) a fine-tuned LLM that generates clarification questions. Our evaluation shows that the fine-tuned LLM outperforms standard zero-shot prompting in generating useful clarification questions. Furthermore, our user study indicates that users find the clarification questions generated by our model to outperform the baseline,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
