AMRG: Extend Vision Language Models for Automatic Mammography Report Generation

TL;DR

This paper introduces AMRG, an end-to-end vision-language model framework for automatic mammography report generation, addressing key challenges with high-resolution images and unstructured language, and establishing a new benchmark in clinical AI.

Contribution

AMRG is the first reproducible, scalable framework for mammography report generation using large VLMs with efficient fine-tuning, and it systematically evaluates multiple models and hyperparameters.

Findings

Achieved ROUGE-L of 0.5691 and BI-RADS accuracy of 0.5582.

Established the first benchmark for mammography report generation.

Demonstrated improved diagnostic consistency and reduced hallucinations.

Abstract

Mammography report generation is a critical yet underexplored task in medical AI, characterized by challenges such as multiview image reasoning, high-resolution visual cues, and unstructured radiologic language. In this work, we introduce AMRG (Automatic Mammography Report Generation), the first end-to-end framework for generating narrative mammography reports using large vision-language models (VLMs). Building upon MedGemma-4B-it-a domain-specialized, instruction-tuned VLM-we employ a parameter-efficient fine-tuning (PEFT) strategy via Low-Rank Adaptation (LoRA), enabling lightweight adaptation with minimal computational overhead. We train and evaluate AMRG on DMID, a publicly available dataset of paired high-resolution mammograms and diagnostic reports. This work establishes the first reproducible benchmark for mammography report generation, addressing a longstanding gap in multimodal clinical AI. We systematically explore LoRA hyperparameter configurations and conduct comparative experiments across multiple VLM backbones, including both domain-specific and general-purpose models under a unified tuning protocol. Our framework demonstrates strong performance across both language generation and clinical metrics, achieving a ROUGE-L score of 0.5691, METEOR of 0.6152, CIDEr of 0.5818, and BI-RADS accuracy of 0.5582. Qualitative analysis further highlights improved diagnostic consistency and reduced hallucinations. AMRG offers a scalable and adaptable foundation for radiology report generation and paves the way for future research in multimodal medical AI.