# Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics

## TL;DR

This paper introduces a reflective-AR display system that enhances the alignment of virtual and real robotic arms in medical surgery, improving user interaction and accuracy during robot setup.

## Contribution

The study presents a novel multi-view reflective-AR display for intuitive and precise virtual-real robot alignment in surgical environments.

## Key findings

- Improved accuracy in virtual-real robot registration with reflective-AR display.
- Enhanced user interaction through multi-view AR visualization.
- Successful application in simulated surgical setup.

## Abstract

Robot-assisted minimally invasive surgery has shown to improve patient outcomes, as well as reduce complications and recovery time for several clinical applications. While increasingly configurable robotic arms can maximize reach and avoid collisions in cluttered environments, positioning them appropriately during surgery is complicated because safety regulations prevent automatic driving. We propose a head-mounted display (HMD) based augmented reality (AR) system designed to guide optimal surgical arm set up. The staff equipped with HMD aligns the robot with its planned virtual counterpart. In this user-centric setting, the main challenge is the perspective ambiguities hindering such collaborative robotic solution. To overcome this challenge, we introduce a novel registration concept for intuitive alignment of AR content to its physical counterpart by providing a multi-view AR experience via reflective-AR displays that simultaneously show the augmentations from multiple viewpoints. Using this system, users can visualize different perspectives while actively adjusting the pose to determine the registration transformation that most closely superimposes the virtual onto the real. The experimental results demonstrate improvement in the interactive alignment of a virtual and real robot when using a reflective-AR display. We also present measurements from configuring a robotic manipulator in a simulated trocar placement surgery using the AR guidance methodology.