e-Health CSIRO at RRG24: Entropy-Augmented Self-Critical Sequence Training for Radiology Report Generation

TL;DR

This paper presents a novel entropy-regularized self-critical sequence training method for radiology report generation from chest X-ray images, achieving top results in the RRG24 challenge.

Contribution

It introduces entropy regularization into self-critical sequence training to enhance diversity and prevent overfitting in radiology report generation models.

Findings

Achieved multiple first-place finishes in RRG24.

Improved vocabulary exploration and report diversity.

Model available on Hugging Face.

Abstract

The Shared Task on Large-Scale Radiology Report Generation (RRG24) aims to expedite the development of assistive systems for interpreting and reporting on chest X-ray (CXR) images. This task challenges participants to develop models that generate the findings and impression sections of radiology reports from CXRs from a patient's study, using five different datasets. This paper outlines the e-Health CSIRO team's approach, which achieved multiple first-place finishes in RRG24. The core novelty of our approach lies in the addition of entropy regularisation to self-critical sequence training, to maintain a higher entropy in the token distribution. This prevents overfitting to common phrases and ensures a broader exploration of the vocabulary during training, essential for handling the diversity of the radiology reports in the RRG24 datasets. Our model is available on Hugging Face https://huggingface.co/aehrc/cxrmate-rrg24.