Evaluating Perceptual Deviations in Video See-Through Head-Mounted Displays while Utilizing Physical Touchscreens

TL;DR

This study evaluates how visual deviations in video see-through head-mounted displays affect real-world interaction, revealing increased positional errors especially at the periphery, with implications for XR usability.

Contribution

It introduces a generalizable measurement method and provides empirical data on perceptual deviations impacting interaction accuracy in XR headsets.

Findings

Significant positional errors in video see-through compared to normal vision.

Errors increase towards the display periphery.

Response times decrease over repeated interactions.

Abstract

Extended reality technology has become a useful tool in many applications, but still suffers from visual deviations that can hamper the utility of the technology. This paper discusses the types of persisting visual deviations experienced when observing the natural world through video see-through head-mounted displays. A generalizable method to measure the effect of these deviations on real-world interaction is designed and used in a human-in-the-loop experiment. The experiment compared video see-through sight through an head-mounted display with normal eyesight in a static set-up, focusing on (camera) lens distortions and display deviations. Participants interacted with a real touchscreen, locating the position of flashed markers shortly after disappearance comparing both conditions to check for deviations in position and time. Results show significant larger mean distance errors between the interaction locations and the original marker positions for video see-through compared to normal eyesight. Moreover, errors increase towards the screen periphery. No significant distance error improvement over time was found, however, response times did significantly decrease for both types of sight.