Object category understanding via eye fixations on freehand sketches

TL;DR

This paper investigates eye fixation patterns on freehand sketches, revealing consistent gaze behaviors across categories, and demonstrates their use in predicting sketch categories and part labels, thus highlighting the potential of gaze data in sketch analysis.

Contribution

It introduces the first large-scale analysis of eye fixations on freehand sketches and develops models for category and part-label prediction based on fixation data.

Findings

Fixation sequences show high consistency within categories.

Gaze data can predict sketch category accurately.

Models can infer part labels from fixations.

Abstract

The study of eye gaze fixations on photographic images is an active research area. In contrast, the image subcategory of freehand sketches has not received as much attention for such studies. In this paper, we analyze the results of a free-viewing gaze fixation study conducted on 3904 freehand sketches distributed across 160 object categories. Our analysis shows that fixation sequences exhibit marked consistency within a sketch, across sketches of a category and even across suitably grouped sets of categories. This multi-level consistency is remarkable given the variability in depiction and extreme image content sparsity that characterizes hand-drawn object sketches. In our paper, we show that the multi-level consistency in the fixation data can be exploited to (a) predict a test sketch's category given only its fixation sequence and (b) build a computational model which predicts part-labels underlying fixations on objects. We hope that our findings motivate the community to deem sketch-like representations worthy of gaze-based studies vis-a-vis photographic images.

Full text

See pages 1-last of output.pdf