Reinforced Medical Report Generation with X-Linear Attention and Repetition Penalty

TL;DR

This paper introduces ReMRG-XR, a novel medical report generation model that leverages x-linear attention for high-order feature interaction and a repetition penalty to reduce redundant terms, significantly improving performance.

Contribution

The work proposes a new model combining x-linear attention and repetition penalty to address limitations of existing methods in medical report generation.

Findings

ReMRG-XR outperforms state-of-the-art baselines on two public datasets.

The model effectively utilizes high-order feature interactions.

Repetition penalty reduces redundant term generation.

Abstract

To reduce doctors' workload, deep-learning-based automatic medical report generation has recently attracted more and more research efforts, where attention mechanisms and reinforcement learning are integrated with the classic encoder-decoder architecture to enhance the performance of deep models. However, these state-of-the-art solutions mainly suffer from two shortcomings: (i) their attention mechanisms cannot utilize high-order feature interactions, and (ii) due to the use of TF-IDF-based reward functions, these methods are fragile with generating repeated terms. Therefore, in this work, we propose a reinforced medical report generation solution with x-linear attention and repetition penalty mechanisms (ReMRG-XR) to overcome these problems. Specifically, x-linear attention modules are used to explore high-order feature interactions and achieve multi-modal reasoning, while repetition penalty is used to apply penalties to repeated terms during the model's training process. Extensive experimental studies have been conducted on two public datasets, and the results show that ReMRG-XR greatly outperforms the state-of-the-art baselines in terms of all metrics.