Auto-US: An Ultrasound Video Diagnosis Agent Using Video Classification Framework and LLMs

TL;DR

Auto-US is an innovative ultrasound diagnosis system combining video classification and large language models, demonstrating high accuracy and clinical relevance in ultrasound video analysis with a new dataset and state-of-the-art methods.

Contribution

The paper introduces Auto-US, a novel AI system integrating ultrasound video classification with diagnostic text generation, supported by a new dataset and advanced neural network models.

Findings

CUV Dataset with 495 videos created for diverse ultrasound analysis

CTU-Net achieves 86.73% accuracy in ultrasound video classification

Auto-US generates clinically meaningful diagnostic suggestions validated by clinicians

Abstract

AI-assisted ultrasound video diagnosis presents new opportunities to enhance the efficiency and accuracy of medical imaging analysis. However, existing research remains limited in terms of dataset diversity, diagnostic performance, and clinical applicability. In this study, we propose \textbf{Auto-US}, an intelligent diagnosis agent that integrates ultrasound video data with clinical diagnostic text. To support this, we constructed \textbf{CUV Dataset} of 495 ultrasound videos spanning five categories and three organs, aggregated from multiple open-access sources. We developed \textbf{CTU-Net}, which achieves state-of-the-art performance in ultrasound video classification, reaching an accuracy of 86.73\% Furthermore, by incorporating large language models, Auto-US is capable of generating clinically meaningful diagnostic suggestions. The final diagnostic scores for each case exceeded 3 out of 5 and were validated by professional clinicians. These results demonstrate the effectiveness and clinical potential of Auto-US in real-world ultrasound applications. Code and data are available at: https://github.com/Bean-Young/Auto-US.