Towards Conditioning Clinical Text Generation for User Control

TL;DR

This paper presents a method using large language models to improve control over clinical text generation, reducing hallucinations and factual errors without increasing clinician workload, and achieves state-of-the-art results on a relevant shared task.

Contribution

It introduces automated dataset augmentation with LLMs as human proxies to enhance clinician control in clinical text generation, achieving significant performance improvements.

Findings

9% relative improvement without augmented training

up to 34% improvement with dataset augmentation

Preliminary human evaluation supports effectiveness

Abstract

Deploying natural language generation systems in clinical settings remains challenging despite advances in Large Language Models (LLMs), which continue to exhibit hallucinations and factual inconsistencies, necessitating human oversight. This paper explores automated dataset augmentation using LLMs as human proxies to condition LLMs for clinician control without increasing cognitive workload. On the BioNLP ACL'24 Discharge Me! Shared Task, we achieve new state-of-the-art results with simpler methods than prior submissions through more efficient training, yielding a 9\% relative improvement without augmented training and up to 34\% with dataset augmentation. Preliminary human evaluation further supports the effectiveness of our approach, highlighting the potential of augmenting clinical text generation for control to enhance relevance, accuracy, and factual consistency.