HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis

TL;DR

This paper introduces HATs, a hierarchical segmentation method for panoramic kidney pathology images that integrates anatomical knowledge and AI foundation models to improve multi-class segmentation accuracy.

Contribution

The novel HATs approach translates spatial relationships into a versatile loss function and leverages EfficientSAM for enhanced segmentation without manual prompts.

Findings

Effective segmentation across 15+ categories

Improved integration of clinical insights

Utilizes EfficientSAM for feature extraction

Abstract

Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel Hierarchical Adaptive Taxonomy Segmentation (HATs) method, which is designed to thoroughly segment panoramic views of kidney structures by leveraging detailed anatomical insights. Our approach entails (1) the innovative HATs technique which translates spatial relationships among 15 distinct object classes into a versatile "plug-and-play" loss function that spans across regions, functional units, and cells, (2) the incorporation of anatomical hierarchies and scale considerations into a unified simple matrix representation for all panoramic entities, (3) the adoption of the latest AI foundation model (EfficientSAM) as a feature extraction tool to boost the model's adaptability, yet eliminating the need for manual prompt generation in conventional segment anything model (SAM). Experimental findings demonstrate that the HATs method offers an efficient and effective strategy for integrating clinical insights and imaging precedents into a unified segmentation model across more than 15 categories. The official implementation is publicly available at https://github.com/hrlblab/HATs.