USPilot: An Embodied Robotic Assistant Ultrasound System with Large Language Model Enhanced Graph Planner
Mingcong Chen, Siqi Fan, Guanglin Cao, Yun-hui Liu, Hongbin Liu

TL;DR
USPilot is an autonomous robotic ultrasound system powered by large language models and graph neural networks, enabling it to understand queries, plan procedures, and perform ultrasound scans, addressing the shortage of professional sonographers.
Contribution
The paper introduces USPilot, a novel system combining LLMs and GNNs for autonomous ultrasound procedures, a significant advancement in robotic medical imaging.
Findings
LLM-enhanced GNN achieves high accuracy in task planning.
USPilot demonstrates effective autonomous ultrasound acquisition.
System responds accurately to ultrasound-related queries.
Abstract
In the era of Large Language Models (LLMs), embodied artificial intelligence presents transformative opportunities for robotic manipulation tasks. Ultrasound imaging, a widely used and cost-effective medical diagnostic procedure, faces challenges due to the global shortage of professional sonographers. To address this issue, we propose USPilot, an embodied robotic assistant ultrasound system powered by an LLM-based framework to enable autonomous ultrasound acquisition. USPilot is designed to function as a virtual sonographer, capable of responding to patients' ultrasound-related queries and performing ultrasound scans based on user intent. By fine-tuning the LLM, USPilot demonstrates a deep understanding of ultrasound-specific questions and tasks. Furthermore, USPilot incorporates an LLM-enhanced Graph Neural Network (GNN) to manage ultrasound robotic APIs and serve as a task planner.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobotics and Automated Systems · Soft Robotics and Applications
