Local Contrastive Learning for Medical Image Recognition

TL;DR

This paper introduces Local Region Contrastive Learning (LRCLR), a novel framework that enhances medical image recognition by focusing on local regions and integrating radiology text, improving interpretability and zero-shot accuracy.

Contribution

LRCLR is a new fine-tuning approach that emphasizes local image regions and cross-modality interaction, addressing limitations of existing self-supervised methods in medical imaging.

Findings

LRCLR identifies significant local regions in chest X-rays.

LRCLR improves zero-shot performance on medical findings.

LRCLR provides meaningful interpretation aligned with radiology reports.

Abstract

The proliferation of Deep Learning (DL)-based methods for radiographic image analysis has created a great demand for expert-labeled radiology data. Recent self-supervised frameworks have alleviated the need for expert labeling by obtaining supervision from associated radiology reports. These frameworks, however, struggle to distinguish the subtle differences between different pathologies in medical images. Additionally, many of them do not provide interpretation between image regions and text, making it difficult for radiologists to assess model predictions. In this work, we propose Local Region Contrastive Learning (LRCLR), a flexible fine-tuning framework that adds layers for significant image region selection as well as cross-modality interaction. Our results on an external validation set of chest x-rays suggest that LRCLR identifies significant local image regions and provides meaningful interpretation against radiology text while improving zero-shot performance on several chest x-ray medical findings.