NucEval: A Robust Evaluation Framework for Nuclear Instance Segmentation

TL;DR

NucEval is a comprehensive evaluation framework for nuclear instance segmentation that addresses key issues in current evaluation methods, improving robustness and reliability in clinical pathology applications.

Contribution

The paper introduces NucEval, a unified framework that tackles evaluation challenges in nuclear segmentation, incorporating solutions for vague regions, score normalization, overlaps, and border uncertainty.

Findings

NucEval improves the robustness of segmentation evaluation metrics.

Applying NucEval significantly impacts the assessment of CNN and ViT models.

The framework is validated on multiple datasets, demonstrating its effectiveness.

Abstract

In computational pathology, nuclear instance segmentation is a fundamental task with many downstream clinical applications. With the advent of deep learning, many approaches, including convolutional neural networks (CNNs) and vision transformers (ViTs), have been proposed for this task, along with both machine learning-based and non-machine learning-based pre- and post-processing techniques to further boost performance. However, one fundamental aspect that has received less attention is the evaluation pipeline. In this study, we identify four key issues associated with nuclear instance segmentation evaluation and propose corresponding solutions. Our proposed modifications, namely handling vague regions, score normalization, overlapping instances, and border uncertainty, are integrated into a unified framework called NucEval, which enables robust evaluation of nuclear instance segmentation. We evaluate this pipeline using the NuInsSeg dataset, which provides unique characteristics that make it particularly suitable for this study, as well as two additional external datasets, with three CNN- and ViT-based nuclear instance segmentation models, to demonstrate the impact of these modifications on instance segmentation metrics. The code, along with complete guidelines and illustrative examples, is publicly available at: https://github.com/masih4/nuc_eval.