Cribriform pattern detection in prostate histopathological images using deep learning models

TL;DR

This study develops and evaluates deep learning models combined with handcrafted features to automatically detect cribriform patterns in prostate histopathological images, aiming to improve consistency and accuracy in cancer grading.

Contribution

The paper introduces a novel automated system using deep neural networks and handcrafted features for cribriform pattern detection, along with an annotated dataset for prostate cancer histopathology.

Findings

Deep learning models outperformed traditional nuclei feature-based methods.

Achieved approximately 86-88% accuracy in cribriform pattern classification.

Provided an annotated dataset for future research in prostate histopathology.

Abstract

Architecture, size, and shape of glands are most important patterns used by pathologists for assessment of cancer malignancy in prostate histopathological tissue slides. Varying structures of glands along with cumbersome manual observations may result in subjective and inconsistent assessment. Cribriform gland with irregular border is an important feature in Gleason pattern 4. We propose using deep neural networks for cribriform pattern classification in prostate histopathological images. $163708$ Hematoxylin and Eosin (H\&E) stained images were extracted from histopathologic tissue slides of $19$ patients with prostate cancer and annotated for cribriform patterns. Our automated image classification system analyses the H\&E images to classify them as either `Cribriform' or `Non-cribriform'. Our system uses various deep learning approaches and hand-crafted image pixel intensity-based features. We present our results for cribriform pattern detection across various parameters and configuration allowed by our system. The combination of fine-tuned deep learning models outperformed the state-of-art nuclei feature based methods. Our image classification system achieved the testing accuracy of $85.93~\pm~7.54$ (cross-validated) and $88.04~\pm~5.63$ ( additional unseen test set) across three folds. In this paper, we present an annotated cribriform dataset along with analysis of deep learning models and hand-crafted features for cribriform pattern detection in prostate histopathological images.