Semantic Scene Graph for Ultrasound Image Explanation and Scanning Guidance
Xuesong Li, Dianye Huang, Yameng Zhang, Nassir Navab, Zhongliang Jiang

TL;DR
This paper introduces a scene graph approach for ultrasound images that explains content and guides scanning, aiming to improve interpretability and usability for non-expert users in point-of-care settings.
Contribution
The study presents a transformer-based scene graph method for ultrasound images combined with LLMs to provide explanations and scanning guidance, enhancing accessibility for non-experts.
Findings
Effective scene graph generation without explicit object detection
Improved ultrasound interpretability for non-experts
Potential to standardize and complete anatomical exploration
Abstract
Understanding medical ultrasound imaging remains a long-standing challenge due to significant visual variability caused by differences in imaging and acquisition parameters. Recent advancements in large language models (LLMs) have been used to automatically generate terminology-rich summaries orientated to clinicians with sufficient physiological knowledge. Nevertheless, the increasing demand for improved ultrasound interpretability and basic scanning guidance among non-expert users, e.g., in point-of-care settings, has not yet been explored. In this study, we first introduce the scene graph (SG) for ultrasound images to explain image content to ordinary and provide guidance for ultrasound scanning. The ultrasound SG is first computed using a transformer-based one-stage method, eliminating the need for explicit object detection. To generate a graspable image explanation for ordinary,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRadiomics and Machine Learning in Medical Imaging · Explainable Artificial Intelligence (XAI) · Medical Imaging and Analysis
