SwEYEpinch: Exploring Intuitive, Efficient Text Entry for Extended Reality via Eye and Hand Tracking

TL;DR

This paper introduces SwEYEpinch, a gaze and pinch-based text entry method for XR that improves speed and user preference through predictive features and sustained learning.

Contribution

It presents a novel gaze-swipe text entry technique with predictive and cancellation features, validated through extensive user studies showing superior performance.

Findings

SwEYEpinch outperforms sequential key selection methods.

Adding mid-swipe prediction improves WPM without reducing accuracy.

Peak performance reaches 64.7 WPM after 30 sessions.

Abstract

Despite steady progress, text entry in Extended Reality (XR) often remains slower and more effortful than typing on a physical keyboard or touchscreen. We explore a simple idea: use gaze to swipe through a virtual keyboard for the fast, low-effort where and a manual pinch held throughout the swipe for the when, extending and validating it through a series of user studies. We first show that a basic version including a low-latency decoder with spatiotemporal Dynamic Time Warping and fixation filtering outperforms selecting individual keys sequentially, either by finger tapping each or gazing at each while pinching. We then add mid-swipe prediction and in-gesture cancellation, improving words per minute (WPM) without hurting accuracy. We show that this approach is faster and more preferred than previous gaze-swipe approaches, finger tapping with prediction, or hand swiping with the same additions. Furthermore, a seven-day, 30-session study demonstrates sustained learning, with peak performance reaching 64.7 WPM.