Remote Assistance with Mixed Reality for Procedural Tasks

TL;DR

This paper introduces a volumetric communication system enabling remote experts to guide local operators in procedural tasks using shared mixed reality views, voice, gestures, and annotations for effective remote assistance.

Contribution

The novel system integrates volumetric communication with mixed reality to enhance remote procedural guidance through shared spatial views and multimodal interaction.

Findings

Effective visualization of shared spatial views using mixed reality glasses

Enhanced remote guidance through voice, gestures, and annotations

Potential improvements in remote procedural task performance

Abstract

We present a volumetric communication system that is designed for remote assistance of procedural tasks. The system allows a remote expert to visually guide a local operator. The two parties share a view that is spatially identical, but for the local operator it is of the object on which they operate, while for the remote expert, the object is presented as a mixed reality "hologram". Guidance is provided by voice, gestures, and annotations performed directly on the object of interest or its hologram. At each end of the communication, spatial is visualized using mixed-reality glasses.