Analysis of Invasive Breast Cancer in Mammograms Using YOLO, Explainability, and Domain Adaptation

TL;DR

This study presents a robust AI framework combining OOD filtering, YOLO detection, and explainability to improve breast cancer detection reliability across diverse imaging modalities.

Contribution

It introduces an integrated approach with ResNet50-based OOD filtering and YOLO architectures, achieving high accuracy and interpretability for mammogram analysis.

Findings

Achieved 99.77% accuracy in OOD detection

Attained 0.947 mAP@0.5 in breast cancer detection

Significantly reduced false alarms on out-of-distribution inputs

Abstract

Deep learning models for breast cancer detection from mammographic images have significant reliability problems when presented with Out-of-Domain (OOD) inputs such as other imaging modalities (CT, MRI, X-ray) or equipment variations, leading to unreliable detection and misdiagnosis. The current research mitigates the fundamental OOD issue through a comprehensive approach integrating ResNet50-based OOD filtering with YOLO architectures (YOLOv8, YOLOv11, YOLOv12) for accurate detection of breast cancer. Our strategy establishes an in-domain gallery via cosine similarity to rigidly reject non-mammographic inputs prior to processing, ensuring that only domain-associated images supply the detection pipeline. The OOD detection component achieves 99.77\% general accuracy with immaculate 100\% accuracy on OOD test sets, effectively eliminating irrelevant imaging modalities. ResNet50 was selected as the optimum backbone after 12 CNN architecture searches. The joint framework unites OOD robustness with high detection performance (mAP@0.5: 0.947) and enhanced interpretability through Grad-CAM visualizations. Experimental validation establishes that OOD filtering significantly improves system reliability by preventing false alarms on out-of-distribution inputs while maintaining higher detection accuracy on mammographic data. The present study offers a fundamental foundation for the deployment of reliable AI-based breast cancer detection systems in diverse clinical environments with inherent data heterogeneity.