Gamified Crowdsourcing for Idiom Corpora Construction

TL;DR

This paper presents a novel gamified crowdsourcing method using a messaging bot to collect idiomatic expressions and usage examples, demonstrating its effectiveness across languages and motivational strategies for building valuable idiom corpora.

Contribution

It introduces the first crowdcreating and crowdrating approach for idiom corpus construction, combining gamification and crowdsourcing in a language-independent manner.

Findings

The approach effectively collects targeted idiomatic data.

Gamification and rewards enhance crowd engagement.

The method accelerates idiom corpus development for multiple applications.

Abstract

Learning idiomatic expressions is seen as one of the most challenging stages in second language learning because of their unpredictable meaning. A similar situation holds for their identification within natural language processing applications such as machine translation and parsing. The lack of high-quality usage samples exacerbates this challenge not only for humans but also for artificial intelligence systems. This article introduces a gamified crowdsourcing approach for collecting language learning materials for idiomatic expressions; a messaging bot is designed as an asynchronous multiplayer game for native speakers who compete with each other while providing idiomatic and nonidiomatic usage examples and rating other players' entries. As opposed to classical crowdprocessing annotation efforts in the field, for the first time in the literature, a crowdcreating & crowdrating approach is implemented and tested for idiom corpora construction. The approach is language independent and evaluated on two languages in comparison to traditional data preparation techniques in the field. The reaction of the crowd is monitored under different motivational means (namely, gamification affordances and monetary rewards). The results reveal that the proposed approach is powerful in collecting the targeted materials, and although being an explicit crowdsourcing approach, it is found entertaining and useful by the crowd. The approach has been shown to have the potential to speed up the construction of idiom corpora for different natural languages to be used as second language learning material, training data for supervised idiom identification systems, or samples for lexicographic studies.