Gaze-Vergence-Controlled See-Through Vision in Augmented Reality

TL;DR

This paper introduces a gaze-vergence-controlled method for see-through AR vision, leveraging gaze depth estimation to improve interaction efficiency and user preference over traditional modalities.

Contribution

It presents a novel gaze-vergence control technique for AR see-through vision, including a gaze tracking module integrated into HoloLens 2 and two control modes for different scenarios.

Findings

Gaze depth estimation is accurate and efficient.

GVC outperforms conventional interaction methods in efficiency.

Users prefer GVC over traditional modalities.

Abstract

Augmented Reality (AR) see-through vision is an interesting research topic since it enables users to see through a wall and see the occluded objects. Most existing research focuses on the visual effects of see-through vision, while the interaction method is less studied. However, we argue that using common interaction modalities, e.g., midair click and speech, may not be the optimal way to control see-through vision. This is because when we want to see through something, it is physically related to our gaze depth/vergence and thus should be naturally controlled by the eyes. Following this idea, this paper proposes a novel gaze-vergence-controlled (GVC) see-through vision technique in AR. Since gaze depth is needed, we build a gaze tracking module with two infrared cameras and the corresponding algorithm and assemble it into the Microsoft HoloLens 2 to achieve gaze depth estimation. We then propose two different GVC modes for see-through vision to fit different scenarios. Extensive experimental results demonstrate that our gaze depth estimation is efficient and accurate. By comparing with conventional interaction modalities, our GVC techniques are also shown to be superior in terms of efficiency and more preferred by users. Finally, we present four example applications of gaze-vergence-controlled see-through vision.