Taming Detection Transformers for Medical Object Detection

TL;DR

This paper explores the application of DETR models for volumetric medical object detection, demonstrating they can match or surpass traditional methods without complex heuristics.

Contribution

It is the first comprehensive evaluation of DETR-based models for 3D medical image detection, showing their competitive performance against established approaches.

Findings

DINO DETR outperforms Retina U-Net on three datasets.

DETR models achieve comparable or better accuracy than existing methods.

Set prediction models simplify the detection pipeline by removing heuristics.

Abstract

The accurate detection of suspicious regions in medical images is an error-prone and time-consuming process required by many routinely performed diagnostic procedures. To support clinicians during this difficult task, several automated solutions were proposed relying on complex methods with many hyperparameters. In this study, we investigate the feasibility of DEtection TRansformer (DETR) models for volumetric medical object detection. In contrast to previous works, these models directly predict a set of objects without relying on the design of anchors or manual heuristics such as non-maximum-suppression to detect objects. We show by conducting extensive experiments with three models, namely DETR, Conditional DETR, and DINO DETR on four data sets (CADA, RibFrac, KiTS19, and LIDC) that these set prediction models can perform on par with or even better than currently existing methods. DINO DETR, the best-performing model in our experiments demonstrates this by outperforming a strong anchor-based one-stage detector, Retina U-Net, on three out of four data sets.