OMG-Net: A Deep Learning Framework Deploying Segment Anything to Detect Pan-Cancer Mitotic Figures from Haematoxylin and Eosin-Stained Slides

TL;DR

This paper introduces OMG-Net, a deep learning framework that leverages the Segment Anything Model to accurately detect mitotic figures across multiple cancer types, improving grading consistency and reducing manual effort.

Contribution

The study establishes the largest pan-cancer mitotic figure dataset and develops OMG-Net, a novel two-stage AI framework that significantly outperforms previous models in MF detection accuracy.

Findings

OMG-Net achieves an F1-score of 0.84 on pan-cancer detection.

Outperforms previous state-of-the-art models by up to 16% F1-score.

Effective across various tumor types and scanner types.

Abstract

Mitotic activity is an important feature for grading several cancer types. Counting mitotic figures (MFs) is a time-consuming, laborious task prone to inter-observer variation. Inaccurate recognition of MFs can lead to incorrect grading and hence potential suboptimal treatment. In this study, we propose an artificial intelligence (AI)-aided approach to detect MFs in digitised haematoxylin and eosin-stained whole slide images (WSIs). Advances in this area are hampered by the limited number and types of cancer datasets of MFs. Here we establish the largest pan-cancer dataset of mitotic figures by combining an in-house dataset of soft tissue tumours (STMF) with five open-source mitotic datasets comprising multiple human cancers and canine specimens (ICPR, TUPAC, CCMCT, CMC and MIDOG++). This new dataset identifies 74,620 MFs and 105,538 mitotic-like figures. We then employed a two-stage framework (the Optimised Mitoses Generator Network (OMG-Net) to classify MFs. The framework first deploys the Segment Anything Model (SAM) to automate the contouring of MFs and surrounding objects. An adapted ResNet18 is subsequently trained to classify MFs. OMG-Net reaches an F1-score of 0.84 on pan-cancer MF detection (breast carcinoma, neuroendocrine tumour and melanoma), largely outperforming the previous state-of-the-art MIDOG++ benchmark model on its hold-out testing set (e.g. +16% F1-score on breast cancer detection, p<0.001) thereby providing superior accuracy in detecting MFs on various types of tumours obtained with different scanners.