Beyond N-grams: A Hierarchical Reward Learning Framework for Clinically-Aware Medical Report Generation

TL;DR

This paper introduces HiMed-RL, a hierarchical reward learning framework for medical report generation that enhances factual accuracy and diagnostic consistency, significantly reducing clinical hallucinations and improving trustworthiness.

Contribution

It proposes a novel hierarchical reward system with dynamic adjustment, explicitly optimizing linguistic fluency, factual grounding, and diagnostic consistency in medical report generation.

Findings

Achieves state-of-the-art performance on multiple benchmarks

Improves out-of-domain report quality by 12.1%

Reduces clinical hallucinations and factual errors

Abstract

Automatic medical report generation can greatly reduce the workload of doctors, but it is often unreliable for real-world deployment. Current methods can write formally fluent sentences but may be factually flawed, introducing serious medical errors known as clinical hallucinations, which make them untrustworthy for diagnosis. To bridge this gap, we introduce HiMed-RL, a Hierarchical Medical Reward Learning Framework designed to explicitly prioritize clinical quality. HiMed-RL moves beyond simple text matching by deconstructing reward learning into three synergistic levels: it first ensures linguistic fluency at the token-level, then enforces factual grounding at the concept-level by aligning key medical terms with expert knowledge, and finally assesses high-level diagnostic consistency at the semantic-level using a specialized LLM verifier. This hierarchical reward is implemented via a Human-inspired Dynamic Reward Adjustment, a strategy which first teaches the model to learn basic facts before progressing to more complex diagnostic reasoning. Experimentally, HiMed-3B achieves state-of-the-art performance on both in-domain and out-of-domain benchmarks, particularly on the latter, with an improvement of 12.1% over the second-best baseline. Our work provides a robust paradigm for generating reports that not only improve fluency but clinical fine-grained quality.