Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation

TL;DR

This paper introduces MCDRL, a novel framework combining causal inference and vision-language models to improve medical image segmentation across diverse domains.

Contribution

It proposes a causal-driven approach leveraging CLIP and text prompts to identify and remove domain-specific confounders, enhancing generalization.

Findings

MCDRL outperforms existing methods in segmentation accuracy.

The framework demonstrates robust generalization across different medical imaging domains.

Extensive experiments validate the effectiveness of causal intervention in medical segmentation.

Abstract

Vision-Language Models (VLMs), such as CLIP, have demonstrated remarkable zero-shot capabilities in various computer vision tasks. However, their application to medical imaging remains challenging due to the high variability and complexity of medical data. Specifically, medical images often exhibit significant domain shifts caused by various confounders, including equipment differences, procedure artifacts, and imaging modes, which can lead to poor generalization when models are applied to unseen domains. To address this limitation, we propose Multimodal Causal-Driven Representation Learning (MCDRL), a novel framework that integrates causal inference with the VLM to tackle domain generalization in medical image segmentation. MCDRL is implemented in two steps: first, it leverages CLIP's cross-modal capabilities to identify candidate lesion regions and construct a confounder dictionary through text prompts, specifically designed to represent domain-specific variations; second, it trains a causal intervention network that utilizes this dictionary to identify and eliminate the influence of these domain-specific variations while preserving the anatomical structural information critical for segmentation tasks. Extensive experiments demonstrate that MCDRL consistently outperforms competing methods, yielding superior segmentation accuracy and exhibiting robust generalizability.