At a Glance to Your Fingertips: Enabling Direct Manipulation of Distant Objects Through SightWarp

TL;DR

SightWarp enables natural, direct manipulation of distant objects in 3D interfaces by using eye-hand coordination to create near-space proxies, improving user performance and interaction immediacy.

Contribution

This paper introduces SightWarp, a novel technique that leverages eye-hand coordination to facilitate direct manipulation of faraway objects in 3D environments.

Findings

Users can easily employ SightWarp for distant object manipulation.

Direct manipulation with SightWarp improves task performance over gaze and pinch methods.

Application examples demonstrate its utility across various 3D interaction scenarios.

Abstract

In 3D user interfaces, reaching out to grab and manipulate something works great until it is out of reach. Indirect techniques like gaze and pinch offer an alternative for distant interaction, but do not provide the same immediacy or proprioceptive feedback as direct gestures. To support direct gestures for faraway objects, we introduce SightWarp, an interaction technique that exploits eye-hand coordination to seamlessly summon object proxies to the user's fingertips. The idea is that after looking at a distant object, users either shift their gaze to the hand or move their hand into view-triggering the creation of a scaled near-space proxy of the object and its surrounding context. The proxy remains active until the eye-hand pattern is released. The key benefit is that users always have an option to immediately operate on the distant object through a natural, direct hand gesture. Through a user study of a 3D object docking task, we show that users can easily employ SightWarp, and that subsequent direct manipulation improves performance over gaze and pinch. Application examples illustrate its utility for 6DOF manipulation, overview-and-detail navigation, and world-in-miniature interaction. Our work contributes to expressive and flexible object interactions across near and far spaces.