Anatomical grounding pre-training for medical phrase grounding

TL;DR

This paper introduces anatomical grounding pre-training for medical phrase grounding, which enhances the alignment of anatomical terms with image regions, significantly improving performance on medical image datasets.

Contribution

It proposes a novel in-domain pre-training task using large-scale datasets to improve medical phrase grounding models.

Findings

Achieved state-of-the-art mIoU of 61.2 on MS-CXR.

Significantly improves zero-shot and fine-tuned MPG performance.

Outperforms existing MPG models with anatomical grounding pre-training.

Abstract

Medical Phrase Grounding (MPG) maps radiological findings described in medical reports to specific regions in medical images. The primary obstacle hindering progress in MPG is the scarcity of annotated data available for training and validation. We propose anatomical grounding as an in-domain pre-training task that aligns anatomical terms with corresponding regions in medical images, leveraging large-scale datasets such as Chest ImaGenome. Our empirical evaluation on MS-CXR demonstrates that anatomical grounding pre-training significantly improves performance in both a zero-shot learning and fine-tuning setting, outperforming state-of-the-art MPG models. Our fine-tuned model achieved state-of-the-art performance on MS-CXR with an mIoU of 61.2, demonstrating the effectiveness of anatomical grounding pre-training for MPG.