CircleFormer: Circular Nuclei Detection in Whole Slide Images with Circle Queries and Attention

TL;DR

CircleFormer is a novel Transformer-based method for detecting and segmenting circular nuclei in medical images, utilizing dynamic anchor circles, circle queries, and a circle cross attention module, achieving promising results on public datasets.

Contribution

The paper introduces CircleFormer, a new approach for circular object detection in medical images that combines circle queries, a circle cross attention module, and a generalized circle IoU loss.

Findings

Achieves promising detection and segmentation performance on MoNuSeg dataset.

Validated effectiveness of each component through ablation studies.

Easily generalizable to segmentation tasks with added branch.

Abstract

Both CNN-based and Transformer-based object detection with bounding box representation have been extensively studied in computer vision and medical image analysis, but circular object detection in medical images is still underexplored. Inspired by the recent anchor free CNN-based circular object detection method (CircleNet) for ball-shape glomeruli detection in renal pathology, in this paper, we present CircleFormer, a Transformer-based circular medical object detection with dynamic anchor circles. Specifically, queries with circle representation in Transformer decoder iteratively refine the circular object detection results, and a circle cross attention module is introduced to compute the similarity between circular queries and image features. A generalized circle IoU (gCIoU) is proposed to serve as a new regression loss of circular object detection as well. Moreover, our approach is easy to generalize to the segmentation task by adding a simple segmentation branch to CircleFormer. We evaluate our method in circular nuclei detection and segmentation on the public MoNuSeg dataset, and the experimental results show that our method achieves promising performance compared with the state-of-the-art approaches. The effectiveness of each component is validated via ablation studies as well. Our code is released at https://github.com/zhanghx-iim-ahu/CircleFormer.