Assessment of Cell Nuclei AI Foundation Models in Kidney Pathology

TL;DR

This study evaluates the performance of three state-of-the-art foundation models for cell nuclei segmentation in kidney pathology, revealing a persistent performance gap and highlighting the need for further model improvements.

Contribution

It provides the largest-scale evaluation of foundation models on diverse kidney pathology datasets, comparing their effectiveness and identifying current limitations.

Findings

CellViT outperformed other models in kidney nuclei segmentation.

All models showed a performance gap in general nuclei segmentation.

The evaluation dataset is the largest of its kind with 2,542 WSIs.

Abstract

Cell nuclei instance segmentation is a crucial task in digital kidney pathology. Traditional automatic segmentation methods often lack generalizability when applied to unseen datasets. Recently, the success of foundation models (FMs) has provided a more generalizable solution, potentially enabling the segmentation of any cell type. In this study, we perform a large-scale evaluation of three widely used state-of-the-art (SOTA) cell nuclei foundation models (Cellpose, StarDist, and CellViT). Specifically, we created a highly diverse evaluation dataset consisting of 2,542 kidney whole slide images (WSIs) collected from both human and rodent sources, encompassing various tissue types, sizes, and staining methods. To our knowledge, this is the largest-scale evaluation of its kind to date. Our quantitative analysis of the prediction distribution reveals a persistent performance gap in kidney pathology. Among the evaluated models, CellViT demonstrated superior performance in segmenting nuclei in kidney pathology. However, none of the foundation models are perfect; a performance gap remains in general nuclei segmentation for kidney pathology.