Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone System

TL;DR

This study investigates the combined use of optical see-through head-mounted displays and smartphones for text entry and output, revealing current challenges that hinder performance and situational awareness in mobile AR systems.

Contribution

It provides empirical insights into the performance limitations of joint OST HMD-smartphone systems for text entry and output in mobile augmented reality contexts.

Findings

Joint system challenges outweigh benefits

Performance issues in text entry and awareness

Need for improved interaction techniques

Abstract

Optical see-through head-mounted displays (OST HMDs) are a popular output medium for mobile Augmented Reality (AR) applications. To date, they lack efficient text entry techniques. Smartphones are a major text entry medium in mobile contexts but attentional demands can contribute to accidents while typing on the go. Mobile multi-display ecologies, such as combined OST HMD-smartphone systems, promise performance and situation awareness benefits over single-device use. We study the joint performance of text entry on mobile phones with text output on optical see-through head-mounted displays. A series of five experiments with a total of 86 participants indicate that, as of today, the challenges in such a joint interactive system outweigh the potential benefits.