Robotic Surgery Remote Mentoring via AR with 3D Scene Streaming and Hand Interaction
Yonghao Long, Chengkun Li, and Qi Dou

TL;DR
This paper presents an AR-based remote mentoring system for robotic surgery that enables intuitive 3D visualization and hand interaction, improving surgical education accessibility and effectiveness.
Contribution
It introduces a novel AR system using HoloLens for real-time 3D scene streaming and hand gesture feedback in robotic surgery training.
Findings
High-fidelity 3D scene streaming validated
Accurate hand gesture feedback demonstrated
Low-latency system performance achieved
Abstract
With the growing popularity of robotic surgery, education becomes increasingly important and urgently needed for the sake of patient safety. However, experienced surgeons have limited accessibility due to their busy clinical schedule or working in a distant city, thus can hardly provide sufficient education resources for novices. Remote mentoring, as an effective way, can help solve this problem, but traditional methods are limited to plain text, audio, or 2D video, which are not intuitive nor vivid. Augmented reality (AR), a thriving technique being widely used for various education scenarios, is promising to offer new possibilities of visual experience and interactive teaching. In this paper, we propose a novel AR-based robotic surgery remote mentoring system with efficient 3D scene visualization and natural 3D hand interaction. Using a head-mounted display (i.e., HoloLens), the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSurgical Simulation and Training · Augmented Reality Applications · Soft Robotics and Applications
