Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts

TL;DR

This paper introduces TextCSP, a hierarchical, text-guided brain tumor segmentation framework that leverages sub-region-specific prompts and a coarse-to-fine decoding process, significantly improving segmentation accuracy.

Contribution

The paper presents a novel hierarchical text-guided segmentation architecture with sub-region-aware prompts and semantic channel modulation, addressing limitations of global text embeddings.

Findings

Achieved 1.7% improvement in Dice score over state-of-the-art methods.

Reduced HD95 metric by 6%, indicating more precise boundary delineation.

Demonstrated consistent performance gains across all tumor sub-regions.

Abstract

Brain tumor segmentation remains challenging because the three standard sub-regions, i.e., whole tumor (WT), tumor core (TC), and enhancing tumor (ET), often exhibit ambiguous visual boundaries. Integrating radiological description texts with imaging has shown promise. However, most multimodal approaches typically compress a report into a single global text embedding shared across all sub-regions, overlooking their distinct clinical characteristics. We propose TextCSP (text-modulated soft cascade architecture), a hierarchical text-guided framework that builds on the TextBraTS baseline with three novel components: (1) a text-modulated soft cascade decoder that predicts WT->TC->ET in a coarse-to-fine manner consistent with their anatomical containment hierarchy. (2) sub-region-aware prompt tuning, which uses learnable soft prompts with a LoRA-adapted BioBERT encoder to generate specialized text representations tailored for each sub-region; (3) text-semantic channel modulators that convert the aforementioned representations into channel-wise refinement signals, enabling the decoder to emphasize features aligned with clinically described patterns. Experiments on the TextBraTS dataset demonstrate consistent improvements across all sub-regions against state-of-the-art methods by 1.7% and 6% on the main metrics Dice and HD95.