DocCHA: Towards LLM-Augmented Interactive Online diagnosis System

TL;DR

DocCHA is a modular, confidence-aware framework that enhances LLM-based clinical diagnosis by enabling adaptive, transparent, and structured multi-turn reasoning, significantly improving diagnostic accuracy and symptom recall.

Contribution

It introduces a novel, interpretable, modular approach to clinical diagnosis with LLMs, incorporating confidence scores for adaptive questioning and reasoning.

Findings

Up to 5.18% higher diagnostic accuracy

Over 30% improvement in symptom recall

Effective in multilingual and resource-constrained settings

Abstract

Despite the impressive capabilities of Large Language Models (LLMs), existing Conversational Health Agents (CHAs) remain static and brittle, incapable of adaptive multi-turn reasoning, symptom clarification, or transparent decision-making. This hinders their real-world applicability in clinical diagnosis, where iterative and structured dialogue is essential. We propose DocCHA, a confidence-aware, modular framework that emulates clinical reasoning by decomposing the diagnostic process into three stages: (1) symptom elicitation, (2) history acquisition, and (3) causal graph construction. Each module uses interpretable confidence scores to guide adaptive questioning, prioritize informative clarifications, and refine weak reasoning links. Evaluated on two real-world Chinese consultation datasets (IMCS21, DX), DocCHA consistently outperforms strong prompting-based LLM baselines (GPT-3.5, GPT-4o, LLaMA-3), achieving up to 5.18 percent higher diagnostic accuracy and over 30 percent improvement in symptom recall, with only modest increase in dialogue turns. These results demonstrate the effectiveness of DocCHA in enabling structured, transparent, and efficient diagnostic conversations -- paving the way for trustworthy LLM-powered clinical assistants in multilingual and resource-constrained settings.