TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking

TL;DR

This paper introduces a webcam-based eye tracking system for crowdsourcing saliency data via Amazon Mechanical Turk, enabling large-scale, cost-effective collection of gaze data comparable to traditional lab methods.

Contribution

It presents a novel, scalable webcam-based eye tracking system and a large saliency dataset, facilitating broader research without specialized hardware.

Findings

Crowdsourced gaze data quality comparable to lab-based data

Significant reduction in cost and effort for data collection

Enabling large-scale saliency dataset creation

Abstract

Traditional eye tracking requires specialized hardware, which means collecting gaze data from many observers is expensive, tedious and slow. Therefore, existing saliency prediction datasets are order-of-magnitudes smaller than typical datasets for other vision recognition tasks. The small size of these datasets limits the potential for training data intensive algorithms, and causes overfitting in benchmark evaluation. To address this deficiency, this paper introduces a webcam-based gaze tracking system that supports large-scale, crowdsourced eye tracking deployed on Amazon Mechanical Turk (AMTurk). By a combination of careful algorithm and gaming protocol design, our system obtains eye tracking data for saliency prediction comparable to data gathered in a traditional lab setting, with relatively lower cost and less effort on the part of the researchers. Using this tool, we build a saliency dataset for a large number of natural images. We will open-source our tool and provide a web server where researchers can upload their images to get eye tracking results from AMTurk.