The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments

TL;DR

This paper introduces a user-friendly tool for creating custom hand gestures for grasping virtual objects, improving naturalness in immersive virtual environments without requiring technical expertise.

Contribution

It presents a novel gesture authoring system using template matching that enables easy creation of object-specific grab gestures for virtual reality.

Findings

Custom gestures are perceived as more natural than pinch or controller-based grasping.

The approach achieves comparable accuracy to existing methods.

User study shows improved naturalness and usability.

Abstract

Natural user interfaces are on the rise. Manufacturers for Augmented, Virtual, and Mixed Reality head mounted displays are increasingly integrating new sensors into their consumer grade products, allowing gesture recognition without additional hardware. This offers new possibilities for bare handed interaction within virtual environments. This work proposes a hand gesture authoring tool for object specific grab gestures allowing virtual objects to be grabbed as in the real world. The presented solution uses template matching for gesture recognition and requires no technical knowledge to design and create custom tailored hand gestures. In a user study, the proposed approach is compared with the pinch gesture and the controller for grasping virtual objects. The different grasping techniques are compared in terms of accuracy, task completion time, usability, and naturalness. The study showed that gestures created with the proposed approach are perceived by users as a more natural input modality than the others.