Deep Networks with Shape Priors for Nucleus Detection

TL;DR

This paper introduces SP-CNN, a novel deep learning approach that incorporates shape priors to improve the accuracy of cell nucleus detection in microscopic images, addressing challenges of shape variability and image quality.

Contribution

The paper presents a new network architecture that integrates domain expert-defined shape priors with CNNs, including a regularization term to reduce false positives and enhance detection within nucleus boundaries.

Findings

SP-CNN outperforms existing methods on challenging datasets.

Incorporating shape priors improves detection accuracy.

The method effectively reduces false positives.

Abstract

Detection of cell nuclei in microscopic images is a challenging research topic, because of limitations in cellular image quality and diversity of nuclear morphology, i.e. varying nuclei shapes, sizes, and overlaps between multiple cell nuclei. This has been a topic of enduring interest with promising recent success shown by deep learning methods. These methods train for example convolutional neural networks (CNNs) with a training set of input images and known, labeled nuclei locations. Many of these methods are supplemented by spatial or morphological processing. We develop a new approach that we call Shape Priors with Convolutional Neural Networks (SP-CNN) to perform significantly enhanced nuclei detection. A set of canonical shapes is prepared with the help of a domain expert. Subsequently, we present a new network structure that can incorporate `expected behavior' of nucleus shapes via two components: {\em learnable} layers that perform the nucleus detection and a {\em fixed} processing part that guides the learning with prior information. Analytically, we formulate a new regularization term that is targeted at penalizing false positives while simultaneously encouraging detection inside cell nucleus boundary. Experimental results on a challenging dataset reveal that SP-CNN is competitive with or outperforms several state-of-the-art methods.