# Gaze-in-wild: A dataset for studying eye and head coordination in everyday activities

## TL;DR

This paper introduces Gaze-in-the-Wild, a novel naturalistic dataset with multimodal eye and head movement data, and evaluates machine learning models for gaze event classification in real-world activities.

## Contribution

The creation of a comprehensive, labeled dataset of naturalistic eye and head movements and the evaluation of machine learning algorithms for gaze event detection in unconstrained environments.

## Key findings

- Classifiers achieve ~90% accuracy for fixations and saccades.
- Detection of pursuit movements remains challenging, with only 60% accuracy.
- Head movement information significantly improves pursuit classification.

## Abstract

The interaction between the vestibular and ocular system has primarily been studied in controlled environments. Consequently, off-the shelf tools for categorization of gaze events (e.g. fixations, pursuits, saccade) fail when head movements are allowed. Our approach was to collect a novel, naturalistic, and multimodal dataset of eye+head movements when subjects performed everyday tasks while wearing a mobile eye tracker equipped with an inertial measurement unit and a 3D stereo camera. This Gaze-in-the-Wild dataset (GW) includes eye+head rotational velocities (deg/s), infrared eye images and scene imagery (RGB+D). A portion was labelled by coders into gaze motion events with a mutual agreement of 0.72 sample based Cohen's $\kappa$. This labelled data was used to train and evaluate two machine learning algorithms, Random Forest and a Recurrent Neural Network model, for gaze event classification. Assessment involved the application of established and novel event based performance metrics. Classifiers achieve $\sim$90$\%$ human performance in detecting fixations and saccades but fall short (60$\%$) on detecting pursuit movements. Moreover, pursuit classification is far worse in the absence of head movement information. A subsequent analysis of feature significance in our best-performing model revealed a reliance upon absolute eye and head velocity, indicating that classification does not require spatial alignment of the head and eye tracking coordinate systems. The GW dataset, trained classifiers and evaluation metrics will be made publicly available with the intention of facilitating growth in the emerging area of head-free gaze event classification.