HUP-3D: A 3D multi-view synthetic dataset for assisted-egocentric hand-ultrasound pose estimation
Manuel Birlo, Razvan Caramalau, Philip J. "Eddie" Edwards, Brian, Dromey, Matthew J. Clarkson, and Danail Stoyanov

TL;DR
HUP-3D is a comprehensive synthetic dataset with multi-view, multi-modal data for hand-ultrasound pose estimation, enabling improved model training and validation in medical and augmented reality applications.
Contribution
The paper introduces HUP-3D, a large synthetic dataset with diverse multi-view and multi-modal data specifically designed for hand-ultrasound pose estimation tasks.
Findings
Achieved lowest hand-object keypoint errors with state-of-the-art models
Demonstrated the dataset's effectiveness in training accurate pose estimation models
Provided a versatile dataset with extensive diversity in views, textures, and lighting
Abstract
We present HUP-3D, a 3D multi-view multi-modal synthetic dataset for hand-ultrasound (US) probe pose estimation in the context of obstetric ultrasound. Egocentric markerless 3D joint pose estimation has potential applications in mixed reality based medical education. The ability to understand hand and probe movements programmatically opens the door to tailored guidance and mentoring applications. Our dataset consists of over 31k sets of RGB, depth and segmentation mask frames, including pose related ground truth data, with a strong emphasis on image diversity and complexity. Adopting a camera viewpoint-based sphere concept allows us to capture a variety of views and generate multiple hand grasp poses using a pre-trained network. Additionally, our approach includes a software-based image rendering concept, enhancing diversity with various hand and arm textures, lighting conditions, and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOrthopedic Surgery and Rehabilitation · Stroke Rehabilitation and Recovery · Musculoskeletal pain and rehabilitation
