MRG-R1: Reinforcement Learning for Clinically Aligned Medical Report Generation

TL;DR

This paper introduces MRG-R1, a reinforcement learning framework that directly optimizes clinical correctness in medical report generation, improving accuracy over traditional token-level training methods.

Contribution

It proposes a semantic-driven reinforcement learning approach with a report-level reward function to enhance medical report accuracy and clinical relevance.

Findings

Improves accuracy and coverage of clinically relevant findings.

Achieves state-of-the-art clinical efficacy on IU X-Ray and MIMIC-CXR datasets.

Outperforms token-level likelihood training methods.

Abstract

Medical report generation aims to automatically produce radiology-style reports from medical images, supporting efficient and accurate clinical decision-making.However, existing approaches predominately rely on token-level likelihood training, which favors local lexical matching and leaves clinical correctness under-specified in the training objective. This behavior can be attributed to token-level likelihood optimization, which rewards surface-form agreement and therefore fails to directly encode constraints on medically accurate findings. To address this objective mismatch, we introduce a semantic-driven reinforcement learning (SRL) framework for medical report generation, named MRG-R1, which directly optimizes report-level clinical correctness rather than token-level likelihood. The key module is a clinically grounded report-level reward function, which reinforces semantic agreement in clinically relevant findings between generated and reference reports, thereby enabling learning signals that explicitly constrain medical correctness beyond surface linguistic alignment. Our evaluations show that the proposed framework improves the accuracy and coverage of clinically relevant findings in generated reports, and that MRG-R1 achieves state-of-the-art clinical efficacy on the IU X-Ray and MIMIC-CXR benchmark datasets.