ChatGPT-4 Vision: a promising tool for diagnosing thyroid nodules

TL;DR

ChatGPT-4 Vision shows potential in diagnosing thyroid nodules with performance comparable to resident physicians, though less accurate than experienced sonographers.

Contribution

This study demonstrates that ChatGPT-4 Vision can achieve diagnostic performance comparable to resident physicians in thyroid nodule classification.

Findings

ChatGPT-4 Vision had a sensitivity of 86.2%, specificity of 60.0%, and AUC of 0.731, comparable to resident physicians but lower than attending physicians.

ChatGPT-4 Vision showed good consistency with resident physicians (Kappa = 0.729) but lower consistency with pathological diagnosis compared to attending physicians.

Abstract

This study aims to evaluate the application of ChatGPT-4 Vision in the ultrasonic image analysis of thyroid nodules by comparing its diagnostic efficacy and consistency with those of sonographers.

In this prospective study, conducted in real clinical scenarios, we included 124 patients with pathologically confirmed thyroid nodules who underwent ultrasound examinations at Fujian Medical University Affiliated Second Hospital. A physician, not involved in the study, collected three ultrasound images for each nodule: the maximum cross-sectional, maximum longitudinal, and the section best representing the nodular characteristics. The images were analyzed by the primed ChatGPT-4 Vision and classified according to the 2020 Chinese Guidelines for Ultrasound Malignancy Risk Stratification of Thyroid Nodules (C-TIRADS). Two sonographers with different qualifications (a resident physician and an attending physician) used the same images to classify the nodules according to the C-TIRADS guidelines. Using fine needle aspiration (FNA) biopsy or surgical pathology results as the gold standard, we compared the consistency and diagnostic efficacy of the primed ChatGPT-4 Vision with those of the sonographers.

(1) ChatGPT-4 Vision diagnosed thyroid nodules with a sensitivity of 86.2%, specificity of 60.0%, and an AUC of 0.731, which was comparable to the resident’s sensitivity of 85.1% (95% CI: 77.2–90.8%), specificity of 66.7% (95% CI: 53.7–77.7%), and AUC of 0.759 (p > 0.05), but lower than the attending physician’s sensitivity of 97.9% (95% CI: 93.2–99.5%), specificity of 80.0% (95% CI: 67.7–88.6%), and AUC of 0.889 (95% CI: 81.5–96.4%) (p < 0.05). (2) The primed ChatGPT-4 Vision demonstrated good consistency with the resident in thyroid nodule classification (Kappa value = 0.729), though its consistency with the pathological diagnosis was lower than that of the attending physician (Kappa values of 0.457 vs. 0.816, respectively).

The primed ChatGPT-4 Vision demonstrates promising clinical utility in thyroid nodule risk stratification, achieving diagnostic performance comparable to resident physicians. Its ability to standardize image analysis aligns with precision medicine goals, offering a foundation for future integration with dynamic ultrasound modalities to enhance pathological correlation.

Figures