CapCLIP: A Vision-Language Representation Alignment Approach for Wireless Capsule Endoscopy Analysis

TL;DR

CapCLIP introduces a vision-language framework for wireless capsule endoscopy that improves zero-shot classification and retrieval by aligning images with clinical text descriptions, enhancing transferability and interpretability.

Contribution

It presents a novel domain-specific vision-language model for WCE that outperforms existing models in zero-shot tasks and enhances semantic understanding.

Findings

CapCLIP outperforms baselines in zero-shot image-text classification.

It achieves strong cross-modal retrieval performance on unseen datasets.

Language-guided learning improves generalisation and interpretability in WCE analysis.

Abstract

Wireless capsule endoscopy (WCE) enables non-invasive visual assessment of the small bowel, but its clinical utility is constrained by the large volume of frames generated per examination and the difficulty of recognising subtle abnormalities under highly variable imaging conditions. Existing learning-based approaches for WCE are predominantly vision-only, often confined to narrow pathology sets, and show limited transfer across datasets and centres. To address these limitations, this study introduces CapCLIP, a domain-specific vision-language representation learning framework for WCE. CapCLIP aligns capsule endoscopy frames with clinically grounded textual descriptions derived from standardised nomenclature and pathology-aware caption templates, thereby learning embeddings that are both semantically informed and transferable. The proposed framework is evaluated against relevant open-source vision and vision-language foundation models under strict zero-shot conditions using unseen WCE datasets. Evaluation covers three downstream tasks: K-nearest neighbour classification, CLIP-style image-text classification, and text-to-image retrieval. Across these settings, CapCLIP consistently outperforms the compared baselines, with particularly strong gains in zero-shot image-text classification and cross-modal retrieval on out-of-distribution datasets. The results indicate that language-guided representation learning can improve both generalisation and semantic interpretability in WCE analysis. These findings position CapCLIP as a step toward foundation models tailored to capsule endoscopy and support the use of language-grounded WCE analysis.