Hierarchical Prototype-based Domain Priors for Multiple Instance Learning in Multimodal Histopathology Analysis

TL;DR

This paper introduces HPDP, a multimodal framework that enhances histopathology analysis by incorporating hierarchical prototypes, spatial encoding, and language-driven semantic alignment, leading to improved accuracy and interpretability.

Contribution

The paper presents a novel hierarchical prototype-based approach with multimodal and language integration for more accurate and interpretable histopathology diagnosis and prognosis.

Findings

HPDP outperforms existing methods across seven cancer cohorts.

The framework improves robustness and interpretability of histopathology analysis.

Hierarchical prototypes and language-driven alignment enhance diagnostic accuracy.

Abstract

Digital pathology has fundamentally altered diagnostic workflows by enabling the computational analysis of gigapixel Whole Slide Images (WSIs), yet effectively deciphering their complex tumor microenvironments remains a formidable challenge. Existing Multiple Instance Learning (MIL) frameworks typically treat Whole Slide Images as unstructured bags of patches, discarding critical morphological semantics and spatial geometry. This lack of inductive bias often leads to overfitting on background noise and fails to align visual features with high-level diagnostic knowledge. To overcome these limitations, we propose the Hierarchical Prototype-based Domain Priors (HPDP) framework, a unified multimodal approach for joint histopathology diagnosis and prognosis. HPDP mitigates the data-driven "black box" issue by introducing a Morphologically Anchored Prototype System (MAPS), which anchors learning to interpretable morphological clusters, and a Sinusoidal Positional Encoder (SPE) to explicitly model tissue architecture. Furthermore, we bridge the semantic gap via a Hierarchical Cross-Modal Alignment (HCMA) module, using Large Language Model (LLM)-generated descriptions to contextually refine visual representations. Extensive experiments across seven cancer cohorts demonstrate that HPDP consistently achieves state-of-the-art performance with superior robustness and interpretability.