Semantic Segmentation for Histopathology using Learned Regularization based on Global Proportions

TL;DR

This paper introduces VSLP, a novel two-stage variational framework that infers detailed tissue segmentation from global proportions without pixel annotations, validated on public and in-house datasets.

Contribution

The paper presents a new variational segmentation method that leverages global label proportions and a pre-trained transformer, improving interpretability and performance over existing weakly supervised approaches.

Findings

Achieves superior performance on public datasets compared to existing methods.

Effectively scales to noisy in-house data, outperforming state-of-the-art techniques.

Provides interpretable segmentation by visualizing the energy components.

Abstract

In pathology, the spatial distribution and proportions of tissue types are key indicators of disease progression, and are more readily available than fine-grained annotations. However, these assessments are rarely mapped to pixel-wise segmentation. The task is fundamentally underdetermined, as many spatially distinct segmentations can satisfy the same global proportions in the absence of pixel-wise constraints. To address this, we introduce Variational Segmentation from Label Proportions (VSLP), a two-stage framework that infers dense segmentations from global label proportions, without any pixel-level annotations. This framework first leverages a pre-trained transformer model with test-time augmentation to produce a pixel-wise confidence estimate. In the second stage, these estimates are fused by solving a variational optimization problem that incorporates a Wasserstein data fidelity term alongside a learned regularizer. Unlike end-to-end networks, our variational method can visualize the fidelity-regularization energy, resulting in more interpretable segmentation. We validate our approach on two public datasets, achieving superior performance over existing weakly supervised and unsupervised methods. For one of these datasets, proportions have been estimated by an experienced pathologist to provide a realistic benchmark to the community. Furthermore, the method scales to an in-house dataset with noisy pathologist labels, severely outperforming state-of-the-art methods, thereby demonstrating practical applicability. The code and data will be made publicly available upon acceptance at https://github.com/xiaoliangpi/VSLP.