Perceptual Requirements for Low-Latency Head-Mounted Displays

TL;DR

This paper introduces Camsicle, a low-latency head-mounted display with 2-millisecond latency, enabling naturalistic studies on how latency affects user experience and performance.

Contribution

The development of Camsicle, a head-mounted display with unprecedented low latency and perspective-correct passthrough, facilitating new perceptual and user experience research.

Findings

Participants preferred lower latencies (2 and 14.3 ms) over higher ones (23 and 29 ms) in a ball-catching task.

Camsicle's low latency allows for naturalistic studies of latency effects on user performance.

Psychophysical thresholds for latency detection relate to subjective preferences in naturalistic scenarios.

Abstract

End-to-end (e2e) latency in head-mounted displays (HMD) is the time delay between a physical change in the world (e.g., a user's head movement) and the moment the display updates to reflect that change. Tracking, rendering, and other computation in real systems invariably introduce some amount of e2e latency to all HMDs. In modern devices this latency is usually in the range of 12-60 milliseconds which is partially addressed through pose prediction and late stage reprojection which means that perceptual studies and user experience evaluations cannot explore latencies below these values. Here, we introduce a video passthrough HMD, called Camsicle, which is capable of 2-millisecond e2e latency and, additionally, uses a catadioptric design to achieve perspective-correct passthrough without reprojection. This platform enables naturalistic user studies to interrogate the impacts of latency on user experience, preference, and performance. Across two user studies and 57 participants we find that 2 and 14.3 millisecond latencies are preferred over 23 and 29 milliseconds when attempting to catch a ball. Additionally, we compare individual latency preferences in this naturalistic ball-catching task to psychophysical thresholds for latency detection in a reference-grade system with zero latency to investigate how psychophysical thresholds may relate to subjective evaluations in naturalistic scenarios.