Quizz: Targeted crowdsourcing with a billion (potential) users
Panagiotis G. Ipeirotis, Evgeniy Gabrilovich

TL;DR
Quizz is a gamified crowdsourcing system that assesses user knowledge and acquires new information by engaging a billion potential users through targeted advertising, enabling cost-effective knowledge curation on niche topics.
Contribution
It introduces a scalable, targeted crowdsourcing approach combining gamification, advertising, and information theory to identify knowledgeable users and gather specialized knowledge efficiently.
Findings
Successfully crowdsourced niche knowledge from over ten thousand users.
Demonstrated cost-effectiveness compared to paid crowdsourcing platforms.
Showed that targeted advertising can identify users with specific expertise.
Abstract
We describe Quizz, a gamified crowdsourcing system that simultaneously assesses the knowledge of users and acquires new knowledge from them. Quizz operates by asking users to complete short quizzes on specific topics; as a user answers the quiz questions, Quizz estimates the user's competence. To acquire new knowledge, Quizz also incorporates questions for which we do not have a known answer; the answers given by competent users provide useful signals for selecting the correct answers for these questions. Quizz actively tries to identify knowledgeable users on the Internet by running advertising campaigns, effectively leveraging the targeting capabilities of existing, publicly available, ad placement services. Quizz quantifies the contributions of the users using information theory and sends feedback to the advertisingsystem about each user. The feedback allows the ad targeting…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
