Mixed reality hologram slicer (mxdR-HS): a marker-less tangible user interface for interactive holographic volume visualization

TL;DR

This paper introduces a marker-less tangible user interface for interactive holographic medical volume visualization using mixed reality headsets, enabling real-time slicing and exploration of volumetric data with minimal latency.

Contribution

The paper presents a novel marker-less planar object tracking method and a hologram slicer interface for mixed reality medical visualization, improving latency and accuracy over existing methods.

Findings

Achieved low computational latency suitable for real-time interaction.

Provided tracking accuracy comparable to marker-based methods.

Enhanced visualization capabilities for complex medical data.

Abstract

Mixed reality head-mounted displays (mxdR-HMD) have the potential to visualize volumetric medical imaging data in holograms to provide a true sense of volumetric depth. An effective user interface, however, has yet to be thoroughly studied. Tangible user interfaces (TUIs) enable a tactile interaction with a hologram through an object. The object has physical properties indicating how it might be used with multiple degrees-of-freedom. We propose a TUI using a planar object (PO) for the holographic medical volume visualization and exploration. We refer to it as mxdR hologram slicer (mxdR-HS). Users can slice the hologram to examine particular regions of interest (ROIs) and intermix complementary data and annotations. The mxdR-HS introduces a novel real-time ad-hoc marker-less PO tracking method that works with any PO where corners are visible. The aim of mxdR-HS is to maintain minimum computational latency while preserving practical tracking accuracy to enable seamless TUI integration in the commercial mxdR-HMD, which has limited computational resources. We implemented the mxdR-HS on a commercial Microsoft HoloLens with a built-in depth camera. Our experimental results showed our mxdR-HS had a superior computational latency but marginally lower tracking accuracy than two marker-based tracking methods and resulted in enhanced computational latency and tracking accuracy than 10 marker-less tracking methods. Our mxdR-HS, in a medical environment, can be suggested as a visual guide to display complex volumetric medical imaging data.