A Novel Augmented Reality Ultrasound Framework Using an RGB-D Camera and a 3D-printed Marker

TL;DR

This paper presents a low-cost augmented reality ultrasound system using an RGB-D camera and a 3D-printed marker, enabling accurate 3D probe tracking without expensive equipment.

Contribution

Developed a simple, calibration-free AR ultrasound framework with a novel registration algorithm and visualization software, suitable for clinical use with consumer-grade devices.

Findings

Achieved probe localization errors of approximately 5.6-5.9 mm.

Demonstrated visually-coherent AR ultrasound overlay in simulated scenarios.

Proposed a cost-effective alternative to traditional tracking systems.

Abstract

Purpose. Ability to locate and track ultrasound images in the 3D operating space is of great benefit for multiple clinical applications. This is often accomplished by tracking the probe using a precise but expensive optical or electromagnetic tracking system. Our goal is to develop a simple and low cost augmented reality echography framework using a standard RGB-D Camera. Methods. A prototype system consisting of an Occipital Structure Core RGB-D camera, a specifically-designed 3D marker, and a fast point cloud registration algorithm FaVoR was developed and evaluated on an Ultrasonix ultrasound system. The probe was calibrated on a 3D-printed N-wire phantom using the software PLUS toolkit. The proposed calibration method is simplified, requiring no additional markers or sensors attached to the phantom. Also, a visualization software based on OpenGL was developed for the augmented reality application. Results. The calibrated probe was used to augment a real-world video in a simulated needle insertion scenario. The ultrasound images were rendered on the video, and visually-coherent results were observed. We evaluated the end-to-end accuracy of our AR US framework on localizing a cube of 5 cm size. From our two experiments, the target pose localization error ranges from 5.6 to 5.9 mm and from -3.9 to 4.2 degrees. Conclusion. We believe that with the potential democratization of RGB-D cameras integrated in mobile devices and AR glasses in the future, our prototype solution may facilitate the use of 3D freehand ultrasound in clinical routine. Future work should include a more rigorous and thorough evaluation, by comparing the calibration accuracy with those obtained by commercial tracking solutions in both simulated and real medical scenarios.