# Mitosis detection in histopathological images using customized deep learning and hybrid optimization algorithms

**Authors:** Afnan M. Alhassan, Nouf I. Altmami

PMC · DOI: 10.1371/journal.pone.0327567 · PLOS One · 2025-07-10

## TL;DR

This paper introduces a customized deep learning model with hybrid optimization algorithms to accurately detect mitosis in cancer histopathological images.

## Contribution

The novel CDL model combines transfer learning, skip connections, and a hybrid optimization algorithm for improved mitosis detection.

## Key findings

- The CDL model achieved an F1 score of 0.994 and 98.8% accuracy on benchmark datasets.
- The model effectively addresses class imbalance and improves localization of mitotic figures.
- Hybrid optimization algorithms enhance the model's momentum and performance.

## Abstract

Identifying mitosis is crucial for cancer diagnosis, but accurate detection remains difficult because of class imbalance and complex morphological variations in histopathological images. To overcome this challenge, we propose a Customized Deep Learning (CDL) model, which integrates advanced deep-learning techniques for better mitosis detection. The CDL model utilizes transfer learning to counter the effects of class imbalance and speed up convergence, while skip connections are also employed to improve the localization of mitosis. Furthermore, we have established an innovative selection mechanism by the hybrid of Jellyfish Search Optimizer (JSO) and Walrus Optimization Algorithm (WOA) to maximize the momentum of the model. The proposed approach is rigorously evaluated on multiple publicly available mitosis detection datasets, including Mitosis WSI CCMCT Training Set, Mitosis-AIC, Mitosis Detection, and Mitosis and Non-Mitosis datasets. To tackle these issues, we hereby bring forth a specifically tailored Custom Deep Learning model, that assimilates hybrid CNN architecture into transfer learning and feature selection for improved mitotic detection. The CDL model comprises a Transfer Learning-based Mitosis Detection module under which extracted features from pre-trained deep networks are used to bolster feature extraction and alleviate class imbalance through skip connections to better localize mitosis. The robust assessment on a benchmark dataset displays the outstanding efficacy of the CDL model, reaching an excellent F1 score of 0.994 and accuracy of 98.8% thus proving its strength for the detection of mitotic figures. This proposed methodology can greatly empower pathologists for accurate appraisal of cancer diagnosis and prognosis. Future lines of exploration will include fusion methodologies and time efficiency for real-time applications, as well as extending CDL to various histopathological analyses.

## Linked entities

- **Diseases:** cancer (MONDO:0004992)

## Full-text entities

- **Diseases:** cancer (MESH:D009369)

## Full text

_Full body text omitted from this summary view._ Fetch the complete paper as Markdown: https://tomesphere.com/paper/PMC12244538/full.md

## Figures

8 figures with captions in the complete paper: https://tomesphere.com/paper/PMC12244538/full.md

## References

35 references — full list in the complete paper: https://tomesphere.com/paper/PMC12244538/full.md

---
Source: https://tomesphere.com/paper/PMC12244538