PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients' Problems and Data Augmentation with Black-box Large Language Models

TL;DR

This paper presents PULSAR, a novel method combining large language models for data augmentation and a specialized pre-training objective to improve automatic summarization of patient problems from medical notes, ranking second in a shared task.

Contribution

We introduce PULSAR, a new approach that integrates data augmentation with LLMs and a novel pre-training task for better summarization of patient problems.

Findings

Our model outperforms larger models by up to 3.1 points on unknown data.

PULSAR is more robust and effective in summarizing patient problems.

Ranked second in the BioNLP 2023 Shared Task 1A.

Abstract

Medical progress notes play a crucial role in documenting a patient's hospital journey, including his or her condition, treatment plan, and any updates for healthcare providers. Automatic summarisation of a patient's problems in the form of a problem list can aid stakeholders in understanding a patient's condition, reducing workload and cognitive bias. BioNLP 2023 Shared Task 1A focuses on generating a list of diagnoses and problems from the provider's progress notes during hospitalisation. In this paper, we introduce our proposed approach to this task, which integrates two complementary components. One component employs large language models (LLMs) for data augmentation; the other is an abstractive summarisation LLM with a novel pre-training objective for generating the patients' problems summarised as a list. Our approach was ranked second among all submissions to the shared task. The performance of our model on the development and test datasets shows that our approach is more robust on unknown data, with an improvement of up to 3.1 points over the same size of the larger model.