Retrieval-Augmented and Knowledge-Grounded Language Models for Faithful Clinical Medicine

TL;DR

This paper introduces Re³Writer, a retrieval-augmented, knowledge-grounded approach that enhances language models to generate faithful, accurate clinical discharge instructions by mimicking physicians' reasoning patterns.

Contribution

The paper presents Re³Writer, a novel method combining retrieval and reasoning to improve the factual accuracy of clinical text generation by language models.

Findings

Significant performance improvements across all metrics.

Enhanced faithfulness and comprehensiveness in generated instructions.

Positive human evaluation results.

Abstract

Language models (LMs), including large language models (such as ChatGPT), have the potential to assist clinicians in generating various clinical notes. However, LMs are prone to produce ``hallucinations'', i.e., generated content that is not aligned with facts and knowledge. In this paper, we propose the Re$^3$Writer method with retrieval-augmented generation and knowledge-grounded reasoning to enable LMs to generate faithful clinical texts. We demonstrate the effectiveness of our method in generating patient discharge instructions. It requires the LMs not to only understand the patients' long clinical documents, i.e., the health records during hospitalization, but also to generate critical instructional information provided both to carers and to the patient at the time of discharge. The proposed Re$^3$Writer imitates the working patterns of physicians to first \textbf{re}trieve related working experience from historical instructions written by physicians, then \textbf{re}ason related medical knowledge. Finally, it \textbf{re}fines the retrieved working experience and reasoned medical knowledge to extract useful information, which is used to generate the discharge instructions for previously-unseen patients. Our experiments show that, using our method, the performance of five representative LMs can be substantially boosted across all metrics. Meanwhile, we show results from human evaluations to measure the effectiveness in terms of fluency, faithfulness, and comprehensiveness.