Token Imbalance Adaptation for Radiology Report Generation

TL;DR

This paper introduces TIMER, a method to address token imbalance in radiology report generation, improving the generation of infrequent medical terms and overall robustness of neural language models.

Contribution

We propose TIMER, a novel approach using unlikelihood loss and reinforcement learning to adaptively improve infrequent token generation in medical report models.

Findings

TIMER outperforms existing methods on IU X-RAY and MIMIC-CXR datasets.

The reinforcement learning component significantly enhances infrequent token generation.

Our approach increases overall model robustness in radiology report generation.

Abstract

Imbalanced token distributions naturally exist in text documents, leading neural language models to overfit on frequent tokens. The token imbalance may dampen the robustness of radiology report generators, as complex medical terms appear less frequently but reflect more medical information. In this study, we demonstrate how current state-of-the-art models fail to generate infrequent tokens on two standard benchmark datasets (IU X-RAY and MIMIC-CXR) of radiology report generation. % However, no prior study has proposed methods to adapt infrequent tokens for text generators feeding with medical images. To solve the challenge, we propose the \textbf{T}oken \textbf{Im}balance Adapt\textbf{er} (\textit{TIMER}), aiming to improve generation robustness on infrequent tokens. The model automatically leverages token imbalance by an unlikelihood loss and dynamically optimizes generation processes to augment infrequent tokens. We compare our approach with multiple state-of-the-art methods on the two benchmarks. Experiments demonstrate the effectiveness of our approach in enhancing model robustness overall and infrequent tokens. Our ablation analysis shows that our reinforcement learning method has a major effect in adapting token imbalance for radiology report generation.