Dataset and Benchmark for Enhancing Critical Retained Foreign Object Detection

TL;DR

This paper introduces the Hopkins RFOs Bench, a large dataset of critical foreign object chest X-rays, benchmarks detection models, and explores synthetic data methods to improve AI detection of rare, critical RFO cases.

Contribution

The paper presents the first large dataset of critical RFO chest X-rays, benchmarks detection models, and evaluates synthetic image generation methods to address data scarcity.

Findings

DeepDRR-RFO and RoentGen-RFO effectively generate realistic synthetic RFO images.

Benchmark results highlight the need for improved detection algorithms for critical RFOs.

Synthetic data can enhance model training for rare critical RFO cases.

Abstract

Critical retained foreign objects (RFOs), including surgical instruments like sponges and needles, pose serious patient safety risks and carry significant financial and legal implications for healthcare institutions. Detecting critical RFOs using artificial intelligence remains challenging due to their rarity and the limited availability of chest X-ray datasets that specifically feature critical RFOs cases. Existing datasets only contain non-critical RFOs, like necklace or zipper, further limiting their utility for developing clinically impactful detection algorithms. To address these limitations, we introduce "Hopkins RFOs Bench", the first and largest dataset of its kind, containing 144 chest X-ray images of critical RFO cases collected over 18 years from the Johns Hopkins Health System. Using this dataset, we benchmark several state-of-the-art object detection models, highlighting the need for enhanced detection methodologies for critical RFO cases. Recognizing data scarcity challenges, we further explore image synthetic methods to bridge this gap. We evaluate two advanced synthetic image methods, DeepDRR-RFO, a physics-based method, and RoentGen-RFO, a diffusion-based method, for creating realistic radiographs featuring critical RFOs. Our comprehensive analysis identifies the strengths and limitations of each synthetic method, providing insights into effectively utilizing synthetic data to enhance model training. The Hopkins RFOs Bench and our findings significantly advance the development of reliable, generalizable AI-driven solutions for detecting critical RFOs in clinical chest X-rays.