Crowd IQ -- Aggregating Opinions to Boost Performance
Michal Kosinski, Yoram Bachrach, Thore Graepel, Giergji Kasneci,, Jurgen Van Gael

TL;DR
This paper introduces Crowd IQ, a method to evaluate and improve decision quality by aggregating crowd opinions on IQ tests, demonstrating rapid growth and saturation with crowd size, and analyzing aggregation methods and individual contributions.
Contribution
It presents Crowd IQ as a novel metric for decision quality based on crowd responses, incorporating probabilistic models and game theory to assess individual and collective performance.
Findings
Crowd IQ increases quickly with crowd size but saturates.
For small homogeneous crowds, Crowd IQ exceeds the IQ of the most intelligent member.
Different aggregation methods impact the resulting Crowd IQ.
Abstract
We show how the quality of decisions based on the aggregated opinions of the crowd can be conveniently studied using a sample of individual responses to a standard IQ questionnaire. We aggregated the responses to the IQ questionnaire using simple majority voting and a machine learning approach based on a probabilistic graphical model. The score for the aggregated questionnaire, Crowd IQ, serves as a quality measure of decisions based on aggregating opinions, which also allows quantifying individual and crowd performance on the same scale. We show that Crowd IQ grows quickly with the size of the crowd but saturates, and that for small homogeneous crowds the Crowd IQ significantly exceeds the IQ of even their most intelligent member. We investigate alternative ways of aggregating the responses and the impact of the aggregation method on the resulting Crowd IQ. We also discuss Contextual…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
