BiCLIP: Bidirectional and Consistent Language-Image Processing for Robust Medical Image Segmentation

TL;DR

BiCLIP is a novel framework that enhances the robustness of medical image segmentation by integrating bidirectional multimodal fusion and augmentation consistency, effectively handling scarce annotations and image artifacts in clinical settings.

Contribution

Introduces BiCLIP, a framework with bidirectional fusion and consistency regularization, improving semantic alignment and robustness in medical image segmentation under challenging conditions.

Findings

Outperforms state-of-the-art baselines on benchmark datasets.

Maintains high performance with only 1% labeled data.

Shows strong resistance to clinical artifacts like motion blur and noise.

Abstract

Medical image segmentation is a cornerstone of computer-assisted diagnosis and treatment planning. While recent multimodal vision-language models have shown promise in enhancing semantic understanding through textual descriptions, their resilience in "in-the-wild" clinical settings-characterized by scarce annotations and hardware-induced image degradations-remains under-explored. We introduce BiCLIP (Bidirectional and Consistent Language-Image Processing), a framework engineered to bolster robustness in medical segmentation. BiCLIP features a bidirectional multimodal fusion mechanism that enables visual features to iteratively refine textual representations, ensuring superior semantic alignment. To further stabilize learning, we implement an augmentation consistency objective that regularizes intermediate representations against perturbed input views. Evaluation on the QaTa-COV19 and MosMedData+ benchmarks demonstrates that BiCLIP consistently surpasses state-of-the-art image-only and multimodal baselines. Notably, BiCLIP maintains high performance when trained on as little as 1% of labeled data and exhibits significant resistance to clinical artifacts, including motion blur and low-dose CT noise.