PalmGazer: Unimanual Eye-hand Menus in Augmented Reality

TL;DR

PalmGazer introduces a novel eye-hand interaction system for augmented reality that enables quick, flexible, and single-handed control of digital commands through a three-way interaction model, enhancing AR usability.

Contribution

The paper presents PalmGazer, a new interaction concept integrating eye and hand gestures for single-handed AR menu control, with design techniques and a prototype demonstrating its effectiveness.

Findings

Simple 2D scroll and selection tasks are easy to perform.

Higher degrees of freedom may require two hands.

The system remains always accessible and movable.

Abstract

How can we design the user interfaces for augmented reality (AR) so that we can interact as simple, flexible and expressive as we can with smartphones in one hand? To explore this question, we propose PalmGazer as an interaction concept integrating eye-hand interaction to establish a singlehandedly operable menu system. In particular, PalmGazer is designed to support quick and spontaneous digital commands -- such as to play a music track, check notifications or browse visual media -- through our devised three-way interaction model: hand opening to summon the menu UI, eye-hand input for selection of items, and dragging gesture for navigation. A key aspect is that it remains always-accessible and movable to the user, as the menu supports meaningful hand and head based reference frames. We demonstrate the concept in practice through a prototypical personal UI with application probes, and describe technique designs specifically-tailored to the application UI. A qualitative evaluation highlights the system's design benefits and drawbacks, e.g., that common 2D scroll and selection tasks are simple to operate, but higher degrees of freedom may be reserved for two hands. Our work contributes interaction techniques and design insights to expand AR's uni-manual capabilities.