Note2Chat: Improving LLMs for Multi-Turn Clinical History Taking Using Medical Notes

TL;DR

Note2Chat introduces a note-driven, multi-stage fine-tuning framework for LLMs that enhances clinical history taking and diagnosis by learning from medical notes, improving interpretability and diagnostic accuracy.

Contribution

The paper presents a novel note-driven training pipeline and a single-turn reasoning paradigm to improve LLMs' performance in multi-turn clinical diagnosis tasks.

Findings

Achieves +16.9 F1 score improvement over GPT-4o.

Achieves +21.0 Top-1 diagnostic accuracy gain.

Demonstrates enhanced interpretability and sample efficiency.

Abstract

Effective clinical history taking is a foundational yet underexplored component of clinical reasoning. While large language models (LLMs) have shown promise on static benchmarks, they often fall short in dynamic, multi-turn diagnostic settings that require iterative questioning and hypothesis refinement. To address this gap, we propose \method{}, a note-driven framework that trains LLMs to conduct structured history taking and diagnosis by learning from widely available medical notes. Instead of relying on scarce and sensitive dialogue data, we convert real-world medical notes into high-quality doctor-patient dialogues using a decision tree-guided generation and refinement pipeline. We then propose a three-stage fine-tuning strategy combining supervised learning, simulated data augmentation, and preference learning. Furthermore, we propose a novel single-turn reasoning paradigm that reframes history taking as a sequence of single-turn reasoning problems. This design enhances interpretability and enables local supervision, dynamic adaptation, and greater sample efficiency. Experimental results show that our method substantially improves clinical reasoning, achieving gains of +16.9 F1 and +21.0 Top-1 diagnostic accuracy over GPT-4o. Our code and dataset can be found at https://github.com/zhentingsheng/Note2Chat.