Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records

TL;DR

This paper explores zero-shot and few-shot prompting strategies using Llama 2 to generate synthetic medical records, achieving high accuracy without training on sensitive patient data, thus addressing privacy concerns in clinical research.

Contribution

It introduces a novel chain-of-thought prompting technique that enables zero-shot LLMs to produce accurate synthetic medical narratives comparable to fine-tuned models.

Findings

Chain-of-thought prompting improves accuracy in synthetic record generation.

Zero-shot approach matches fine-tuned models in Rouge metrics.

Method enhances privacy-preserving data synthesis for medical research.

Abstract

The challenge of accessing historical patient data for clinical research, while adhering to privacy regulations, is a significant obstacle in medical science. An innovative approach to circumvent this issue involves utilising synthetic medical records that mirror real patient data without compromising individual privacy. The creation of these synthetic datasets, particularly without using actual patient data to train Large Language Models (LLMs), presents a novel solution as gaining access to sensitive patient information to train models is also a challenge. This study assesses the capability of the Llama 2 LLM to create synthetic medical records that accurately reflect real patient information, employing zero-shot and few-shot prompting strategies for comparison against fine-tuned methodologies that do require sensitive patient data during training. We focus on generating synthetic narratives for the History of Present Illness section, utilising data from the MIMIC-IV dataset for comparison. In this work introduce a novel prompting technique that leverages a chain-of-thought approach, enhancing the model's ability to generate more accurate and contextually relevant medical narratives without prior fine-tuning. Our findings suggest that this chain-of-thought prompted approach allows the zero-shot model to achieve results on par with those of fine-tuned models, based on Rouge metrics evaluation.