NoteChat: A Dataset of Synthetic Doctor-Patient Conversations Conditioned on Clinical Notes

TL;DR

NoteChat is a multi-agent LLM framework that generates high-quality synthetic patient-physician dialogues conditioned on clinical notes, improving clinical note generation and potentially aiding patient engagement and reducing physician burnout.

Contribution

It introduces a novel multi-agent role-play approach with structured prompting to generate synthetic dialogues, enhancing clinical note modeling beyond existing methods.

Findings

Outperforms state-of-the-art models like ChatGPT and GPT-4 in dialogue quality.

Achieves up to 22.78% improvement in domain expert evaluations.

Enhances clinical note generation and supports patient engagement.

Abstract

We introduce NoteChat, a novel cooperative multi-agent framework leveraging Large Language Models (LLMs) to generate patient-physician dialogues. NoteChat embodies the principle that an ensemble of role-specific LLMs, through structured role-play and strategic prompting, can perform their assigned roles more effectively. The synergy among these role-playing LLMs results in a cohesive and efficient dialogue generation. Evaluation on MTS-dialogue, a benchmark dataset for patient-physician dialogues-note pairs, shows that models trained with the augmented synthetic patient-physician dialogues by NoteChat outperforms other state-of-the-art models for generating clinical notes. Our comprehensive automatic and human evaluation demonstrates that NoteChat substantially surpasses state-of-the-art models like ChatGPT and GPT-4 up to 22.78% by domain experts in generating superior synthetic patient-physician dialogues based on clinical notes. NoteChat has the potential to engage patients directly and help clinical documentation, a leading cause of physician burnout.