Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided Explorations

TL;DR

This paper presents a learning-based framework for robotic ultrasound scanning that combines imitation learning from human demonstrations with guided exploration to improve autonomous ultrasound performance.

Contribution

It introduces a multi-modal model capturing ultrasound images, probe pose, and contact force, and combines imitation learning with post-optimization for enhanced robotic ultrasound skills.

Findings

Validated the framework through robotic experiments

Achieved improved scanning accuracy and reliability

Demonstrated effective learning from experienced physicians

Abstract

Medical ultrasound has become a routine examination approach nowadays and is widely adopted for different medical applications, so it is desired to have a robotic ultrasound system to perform the ultrasound scanning autonomously. However, the ultrasound scanning skill is considerably complex, which highly depends on the experience of the ultrasound physician. In this paper, we propose a learning-based approach to learn the robotic ultrasound scanning skills from human demonstrations. First, the robotic ultrasound scanning skill is encapsulated into a high-dimensional multi-modal model, which takes the ultrasound images, the pose/position of the probe and the contact force into account. Second, we leverage the power of imitation learning to train the multi-modal model with the training data collected from the demonstrations of experienced ultrasound physicians. Finally, a post-optimization procedure with guided explorations is proposed to further improve the performance of the learned model. Robotic experiments are conducted to validate the advantages of our proposed framework and the learned models.