ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning

TL;DR

This paper introduces ALFA, a framework that enhances large language models' ability to ask effective questions by decomposing question quality into attributes, synthesizing variations, and aligning models through preference optimization, demonstrated in clinical reasoning.

Contribution

ALFA provides a novel, attribute-based approach to improve LLM question-asking, with a case study in healthcare that shows significant reduction in diagnostic errors.

Findings

Models aligned with ALFA reduce diagnostic errors by 56.6%.

Achieved a question-level win-rate of 64.4%.

Demonstrated strong generalizability across tasks.

Abstract

Large language models (LLMs) often fail to ask effective questions under uncertainty, making them unreliable in domains where proactive information-gathering is essential for decision-making. We present ALignment via Fine-grained Attributes, (ALFA) a framework that improves LLM question-asking by (i) decomposing the notion of a "good" question into a set of theory-grounded attributes (e.g., clarity, relevance), (ii) controllably synthesizing attribute-specific question variations, and (iii) aligning models via preference-based optimization to explicitly learn to ask better questions along these fine-grained attributes. Focusing on clinical reasoning as a case study, we introduce the MediQ-AskDocs dataset, composed of 17k real-world clinical interactions augmented with 80k attribute-specific preference pairs of follow-up questions, as well as a novel expert-annotated interactive healthcare QA task to evaluate question-asking abilities. Models aligned with ALFA reduce diagnostic errors by 56.6% on MediQ-AskDocs compared to SoTA instruction-tuned LLMs, with a question-level win-rate of 64.4% and strong generalizability. Our findings suggest that explicitly guiding question-asking with structured, fine-grained attributes offers a scalable path to improve LLMs, especially in expert application domains.