MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

TL;DR

MedGround introduces a scalable pipeline and dataset for improving medical vision-language models by providing verified grounding data, leading to better clinical narrative grounding and generalization.

Contribution

We develop MedGround, an automated method to generate high-quality medical grounding data from segmentation resources, enhancing VLM training and performance.

Findings

Improved grounding accuracy in VLMs trained with MedGround-35K

Enhanced multi-object semantic disambiguation

Strong generalization to unseen grounding scenarios

Abstract

Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We posit this limitation arises from the scarcity of high-quality, large-scale clinical referring-localization pairs. To address this, we introduce MedGround, an automated pipeline that transforms segmentation resources into high-quality medical referring grounding data. Leveraging expert masks as spatial anchors, MedGround precisely derives localization targets, extracts shape and spatial cues, and guides VLMs to synthesize natural, clinically grounded queries that reflect morphology and location. To ensure data rigor, a multi-stage verification system integrates strict formatting checks, geometry- and medical-prior rules, and image-based visual judging to filter out ambiguous or visually unsupported samples. Finally, we present MedGround-35K, a novel multimodal medical dataset. Extensive experiments demonstrate that VLMs trained with MedGround-35K consistently achieve improved referring grounding performance, enhance multi-object semantic disambiguation, and exhibit strong generalization to unseen grounding settings. This work highlights MedGround as a scalable, data-driven approach to anchor medical language to verifiable visual evidence. Dataset and code will be released publicly upon acceptance.