# eyeNotate: Interactive Annotation of Mobile Eye Tracking Data Based on Few-Shot Image Classification

## TL;DR

eyeNotate is a web-based tool that helps annotate mobile eye tracking data more efficiently using interactive and semi-automatic methods.

## Contribution

The tool introduces a few-shot image classification model to improve annotation efficiency and usability.

## Key findings

- The IML-support version improved annotation efficiency compared to the baseline.
- Expert annotators found the tool usable and reliable for mapping fixations to AOIs.
- Three image classification models were tested for performance on remaining data.

## Abstract

Mobile eye tracking is an important tool in psychology and human-centered interaction design for understanding how people process visual scenes and user interfaces. However, analyzing recordings from head-mounted eye trackers, which typically include an egocentric video of the scene and a gaze signal, is a time-consuming and largely manual process. To address this challenge, we develop eyeNotate, a web-based annotation tool that enables semi-automatic data annotation and learns to improve from corrective user feedback. Users can manually map fixation events to areas of interest (AOIs) in a video-editing-style interface (baseline version). Further, our tool can generate fixation-to-AOI mapping suggestions based on a few-shot image classification model (IML-support version). We conduct an expert study with trained annotators (n = 3) to compare the baseline and IML-support versions. We measure the perceived usability, annotations’ validity and reliability, and efficiency during a data annotation task. We asked our participants to re-annotate data from a single individual using an existing dataset (n = 48). Further, we conducted a semi-structured interview to understand how participants used the provided IML features and assessed our design decisions. In a post hoc experiment, we investigate the performance of three image classification models in annotating data of the remaining 47 individuals.