A Study of Performance and Interaction Patterns in Hand and Tangible Interaction in Tabletop Mixed Reality

TL;DR

This study compares hand and tangible interactions for 3D object manipulation in mixed reality, revealing tangible interfaces offer higher precision despite increased errors in complex tasks.

Contribution

It provides a detailed analysis of hand versus tangible interaction performance in MR, highlighting the advantages of tangible proxies for precision in 4DoF tasks.

Findings

Tangible interactions outperform hand interactions in precision.

Compound tasks increase correction efforts and errors.

Tangible interactions show larger errors but higher accuracy overall.

Abstract

This paper presents a comprehensive study of virtual 3D object manipulation along 4DoF on real surfaces in mixed reality (MR), using hand-based and tangible interactions. A custom cylindrical tangible proxy leverages affordances of physical knobs and tabletop support for stable input. We evaluate both modalities across isolated tasks (2DoF translation, 1DoF rotation scaling), semicombined (3DoF translation rotation), and full 4DoF compound manipulation. We offer analyses of hand interactions, tangible interactions, and their comparison in MR tasks. For hand interactions, compound tasks required repetitive corrections, increasing completion times yet surprisingly, rotation errors were smaller in compound tasks than in rotation only tasks. Tangible interactions exhibited significantly larger errors in translation, rotation, and scaling during compound tasks compared to isolated tasks. Crucially, tangible interactions outperformed hand interactions in precision, likely due to tabletop support and constrained 4DoF design. These findings inform designers opting for hand-only interaction (highlighting tradeoffs in compound tasks) and those leveraging tangibles (emphasizing precision gains despite compound-task challenges).