Surgi-HDTMR: Closing the Sensorimotor Loop in Bimanual Microsurgery via Haptics, Digital Twin, and Mixed Reality

TL;DR

Surgi-HDTMR introduces a mixed-reality and digital twin system with adaptive haptics for enhanced microsurgical training, improving speed, safety, and perceptual accuracy in simulated procedures.

Contribution

This work presents a novel integrated MR, DT, and haptic system that enhances microsurgical training by providing immersive, synchronized feedback and guidance, which was not available in prior platforms.

Findings

Reduced task completion time in simulated surgeries

Lower incidence of harmful contacts and collisions

Enhanced perceptual accuracy during training

Abstract

Robotic microsurgery demands precise bimanual control, intuitive interaction, and informative force feedback. However, most training platforms for robotic microsurgery lack immersive 3D interaction and high-fidelity haptics. Here, we present Surgi-HDTMR, a mixed-reality (MR) and digital-twin (DT) training system that couples bimanual haptic teleoperation with a benchtop microsurgical robotic platform, and 3D-printed phantoms. A metrically co-registered, time-synchronized DT aligns in-situ MR guidance with the physical workspace and drives a depth-adaptive haptic model that renders contact, puncture, and tissue-retraction forces. In a within-subjects study of simulated cortical navigation and tumor resection, Surgi-HDTMR shortened task time, reduced harmful contacts and collisions, and improved perceptual accuracy relative to non-haptic and non-adaptive baselines. These results suggest that tightly coupling MR overlays with a synchronized DT, together with depth-adaptive haptics, can accelerate skill acquisition and improve safety in robot-assisted microsurgery, pointing toward next-generation surgical training.