Text-Driven Weakly Supervised OCT Lesion Segmentation with Structural Guidance

TL;DR

This paper introduces a novel weakly supervised OCT lesion segmentation method that combines structural and text-driven guidance to achieve high-quality pixel-level segmentation using only image-level labels.

Contribution

It presents a multi-modal framework integrating visual and textual cues, leveraging large-scale pretrained models to improve lesion segmentation in OCT images with minimal supervision.

Findings

Achieves state-of-the-art segmentation accuracy on three OCT datasets.

Effectively utilizes structural and textual guidance to enhance lesion localization.

Reduces annotation effort by relying solely on image-level labels.

Abstract

Accurate segmentation of Optical Coherence Tomography (OCT) images is crucial for diagnosing and monitoring retinal diseases. However, the labor-intensive nature of pixel-level annotation limits the scalability of supervised learning for large datasets. Weakly Supervised Semantic Segmentation (WSSS) offers a promising alternative by using weaker forms of supervision, such as image-level labels, to reduce the annotation burden. Despite its advantages, weak supervision inherently carries limited information. We propose a novel WSSS framework with only image-level labels for OCT lesion segmentation that integrates structural and text-driven guidance to produce high-quality, pixel-level pseudo labels. The framework employs two visual processing modules: one that processes the original OCT images and another that operates on layer segmentations augmented with anomalous signals, enabling the model to associate lesions with their corresponding anatomical layers. Complementing these visual cues, we leverage large-scale pretrained models to provide two forms of textual guidance: label-derived descriptions that encode local semantics, and domain-agnostic synthetic descriptions that, although expressed in natural image terms, capture spatial and relational semantics useful for generating globally consistent representations. By fusing these visual and textual features in a multi-modal framework, our method aligns semantic meaning with structural relevance, thereby improving lesion localization and segmentation performance. Experiments on three OCT datasets demonstrate state-of-the-art results, highlighting its potential to advance diagnostic accuracy and efficiency in medical imaging.