RadBARTsum: Domain Specific Adaption of Denoising Sequence-to-Sequence Models for Abstractive Radiology Report Summarization

TL;DR

RadBARTsum is a domain-specific adaptation of the BART model that uses entity masking and fine-tuning on radiology reports to improve automatic report summarization, aiding quicker clinical decision-making.

Contribution

This work introduces RadBARTsum, a novel domain-specific generative model for radiology report summarization utilizing entity masking and targeted fine-tuning.

Findings

Entity masking improves model performance.

Domain-specific training enhances summarization accuracy.

Model effectively captures clinical knowledge.

Abstract

Radiology report summarization is a crucial task that can help doctors quickly identify clinically significant findings without the need to review detailed sections of reports. This study proposes RadBARTsum, a domain-specific and ontology facilitated adaptation of the BART model for abstractive radiology report summarization. The approach involves two main steps: 1) re-training the BART model on a large corpus of radiology reports using a novel entity masking strategy to improving biomedical domain knowledge learning, and 2) fine-tuning the model for the summarization task using the Findings and Background sections to predict the Impression section. Experiments are conducted using different masking strategies. Results show that the re-training process with domain knowledge facilitated masking improves performances consistently across various settings. This work contributes a domain-specific generative language model for radiology report summarization and a method for utilising medical knowledge to realise entity masking language model. The proposed approach demonstrates a promising direction of enhancing the efficiency of language models by deepening its understanding of clinical knowledge in radiology reports.