SentenceRacer: A Game with a Purpose for Image Sentence Annotation

TL;DR

SentenceRacer is an online game that engages users in generating and verifying accurate image descriptions, reducing costs and improving annotation quality compared to traditional crowdsourcing methods.

Contribution

It introduces a gamified approach for collecting and validating image sentence annotations at no cost, enhancing data quality over existing crowdsourcing techniques.

Findings

Higher quality annotations than Amazon Mechanical Turk

Cost-effective data collection method

Engages users through an interactive game

Abstract

Recently datasets that contain sentence descriptions of images have enabled models that can automatically generate image captions. However, collecting these datasets are still very expensive. Here, we present SentenceRacer, an online game that gathers and verifies descriptions of images at no cost. Similar to the game hangman, players compete to uncover words in a sentence that ultimately describes an image. SentenceRacer both generates and verifies that the sentences are accurate descriptions. We show that SentenceRacer generates annotations of higher quality than those generated on Amazon Mechanical Turk (AMT).