Bend-Net: Bending Loss Regularized Multitask Learning Network for Nuclei Segmentation in Histopathology Images

TL;DR

Bend-Net is a novel multitask learning network with a bending loss regularizer designed to improve the segmentation of overlapped nuclei in histopathology images, addressing a key challenge in medical image analysis.

Contribution

The paper introduces a new multitask architecture with a bending loss that enhances separation of overlapped nuclei and proposes two new metrics for evaluation.

Findings

Bend-Net outperforms eight state-of-the-art methods on multiple datasets.

The bending loss effectively penalizes concave contours to improve segmentation.

New metrics AJIO and ACCO provide better evaluation of overlapped nuclei segmentation.

Abstract

Separating overlapped nuclei is a major challenge in histopathology image analysis. Recently published approaches have achieved promising overall performance on nuclei segmentation; however, their performance on separating overlapped nuclei is quite limited. To address the issue, we propose a novel multitask learning network with a bending loss regularizer to separate overlapped nuclei accurately. The newly proposed multitask learning architecture enhances the generalization by learning shared representation from three tasks: instance segmentation, nuclei distance map prediction, and overlapped nuclei distance map prediction. The proposed bending loss defines high penalties to concave contour points with large curvatures, and applies small penalties to convex contour points with small curvatures. Minimizing the bending loss avoids generating contours that encompass multiple nuclei. In addition, two new quantitative metrics, Aggregated Jaccard Index of overlapped nuclei (AJIO) and Accuracy of overlapped nuclei (ACCO), are designed for the evaluation of overlapped nuclei segmentation. We validate the proposed approach on the CoNSeP and MoNuSegv1 datasets using seven quantitative metrics: Aggregate Jaccard Index, Dice, Segmentation Quality, Recognition Quality, Panoptic Quality, AJIO, and ACCO. Extensive experiments demonstrate that the proposed Bend-Net outperforms eight state-of-the-art approaches.