UniSemAlign: Text-Prototype Alignment with a Foundation Encoder for Semi-Supervised Histopathology Segmentation

TL;DR

UniSemAlign introduces a novel semi-supervised histopathology segmentation framework that leverages class-level structure through prototype and text alignment, significantly improving performance with limited labeled data.

Contribution

It presents a dual-modal alignment approach using a pathology-pretrained Transformer encoder to enhance semi-supervised segmentation accuracy.

Findings

Achieves up to 2.6% Dice improvement on GlaS with 10% labels.

Achieves up to 8.6% Dice improvement on CRAG with 10% labels.

Outperforms recent semi-supervised baselines on two datasets.

Abstract

Semi-supervised semantic segmentation in computational pathology remains challenging due to scarce pixel-level annotations and unreliable pseudo-label supervision. We propose UniSemAlign, a dual-modal semantic alignment framework that enhances visual segmentation by injecting explicit class-level structure into pixel-wise learning. Built upon a pathology-pretrained Transformer encoder, UniSemAlign introduces complementary prototype-level and text-level alignment branches in a shared embedding space, providing structured guidance that reduces class ambiguity and stabilizes pseudo-label refinement. The aligned representations are fused with visual predictions to generate more reliable supervision for unlabeled histopathology images. The framework is trained end-to-end with supervised segmentation, cross-view consistency, and cross-modal alignment objectives. Extensive experiments on the GlaS and CRAG datasets demonstrate that UniSemAlign substantially outperforms recent semi-supervised baselines under limited supervision, achieving Dice improvements of up to 2.6% on GlaS and 8.6% on CRAG with only 10% labeled data, and strong improvements at 20% supervision. Code is available at: https://github.com/thailevann/UniSemAlign