Augmenting Mobile Phone Interaction with Face-Engaged Gestures
Jian Zhao, Ricardo Jota, Daniel Wigdor, Ravin Balakrishnan

TL;DR
This paper explores face-engaged gestures detected via smartphone front cameras, combining head movements with touch and motion inputs to expand mobile interaction possibilities, including touch-free and one-handed use.
Contribution
Introduces novel face-engaged interaction techniques that augment traditional mobile inputs by leveraging head movements with touch and device sensors.
Findings
Users reacted positively to the new interaction techniques.
The techniques enable touch-free and one-handed interactions.
The methods expand the mobile interaction vocabulary.
Abstract
The movement of a user's face, easily detected by a smartphone's front camera, is an underexploited input modality for mobile interactions. We introduce three sets of face-engaged interaction techniques for augmenting the traditional mobile inputs, which leverages the combination of the head movements with touch gestures and device motions, all sensed via the phone's built-in sensors. We systematically present the space of design considerations for mobile interactions using one or more of the three input modalities (i.e., touch, motion, and head). The additional affordances of the proposed techniques expand the mobile interaction vocabulary, and can facilitate unique usage scenarios such as one-hand or touch-free interaction. An initial evaluation was conducted and users had positive reactions to the new techniques, indicating the promise of an intuitive and convenient user experience.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInteractive and Immersive Displays · Gaze Tracking and Assistive Technology · Tactile and Sensory Interactions
