Optimizing Artificial Intelligence Thresholds for Mammographic Lesion Detection: A Retrospective Study on Diagnostic Performance and Radiologist–Artificial Intelligence Discordance

TL;DR

This study shows that adjusting AI thresholds based on lesion categories improves breast cancer detection accuracy in mammograms.

Contribution

The paper introduces category-specific AI thresholds for BI-RADS 4A and 4B/4C lesions to enhance diagnostic performance.

Findings

AI scores were significantly higher for malignant versus benign cases (72.1 vs. 20.9; p < 0.001).

Optimal thresholds of 19 for BI-RADS 4A and 63 for BI-RADS 4B/4C improved diagnostic accuracy.

Abstract

Background/Objectives: Artificial intelligence (AI)-based systems are increasingly being used to assist radiologists in detecting breast cancer on mammograms. However, applying fixed AI score thresholds across diverse lesion types may compromise diagnostic performance, especially in women with dense breasts. This study aimed to determine optimal category-specific AI thresholds and to analyze discrepancies between AI predictions and radiologist assessments, particularly for BI-RADS 4A versus 4B/4C lesions. Methods: We retrospectively analyzed 194 mammograms (76 BI-RADS 4A and 118 BI-RADS 4B/4C) using FDA-approved AI software. Lesion characteristics, breast density, AI scores, and pathology results were collected. A receiver operating characteristic (ROC) analysis was conducted to determine the optimal thresholds via Youden’s index. Discrepancy analysis focused on BI-RADS 4A lesions with AI scores of ≥35 and BI-RADS 4B/4C lesions with AI scores of <35. Results: AI scores were significantly higher in malignant versus benign cases (72.1 vs. 20.9; p < 0.001). The optimal AI threshold was 19 for BI-RADS 4A (AUC = 0.685) and 63 for BI-RADS 4B/4C (AUC = 0.908). In discordant cases, BI-RADS 4A lesions with scores of ≥35 had a malignancy rate of 43.8%, while BI-RADS 4B/4C lesions with scores of <35 had a malignancy rate of 19.5%. Conclusions: Using category-specific AI thresholds improves diagnostic accuracy and supports radiologist decision-making. However, limitations persist in BI-RADS 4A cases with overlapping scores, reinforcing the need for radiologist oversight and tailored AI integration strategies in clinical practice.

Figures