WigglyEyes: Inferring Eye Movements from Keypress Data

TL;DR

This paper introduces a model that infers users' eye movements during interaction solely from keypress data, providing a cost-effective alternative to traditional eye tracking methods.

Contribution

It presents a novel inference architecture that personalizes gaze prediction using user-specific parameters and a new loss function for better synchronization with keypresses.

Findings

Accurately infers gaze from keypress data during touchscreen typing

Personalized model improves inference accuracy

Provides a practical proxy for eye tracking in data-scarce scenarios

Abstract

We present a model for inferring where users look during interaction based on keypress data only. Given a key log, it outputs a scanpath that tells, moment-by-moment, how the user had moved eyes while entering those keys. The model can be used as a proxy for human data in cases where collecting real eye tracking data is expensive or impossible. Our technical insight is an inference architecture that considers the individual characteristics of the user, inferred as a low-dimensional parameter vector. We present a novel loss function for synchronizing inferred eye movements with the keypresses. Evaluations on touchscreen typing demonstrate accurate gaze inference.