Competition and Collaboration in Crowdsourcing Communities: What happens when peers evaluate each other?

TL;DR

This study investigates how peer evaluations in crowdsourcing communities balance competitive and collaborative motives, revealing that skill level influences evaluation strategies and impacts community participation and structure.

Contribution

It provides empirical evidence on the dynamics of peer evaluation behavior and its effects on community sustainability in crowdsourcing platforms.

Findings

High-skilled members become more competitive and sabotage close competitors.

High-skilled members show leniency towards less threatening peers.

Sabotage reduces future participation of low-skill targets.

Abstract

Crowdsourcing has evolved as an organizational approach to distributed problem solving and innovation. As contests are embedded in online communities and evaluation rights are assigned to the crowd, community members face a tension: they find themselves exposed to both competitive motives to win the contest prize and collaborative participation motives in the community. The competitive motive suggests they may evaluate rivals strategically according to their self-interest, the collaborative motive suggests they may evaluate their peers truthfully according to mutual interest. Using field data from Threadless on 38 million peer evaluations of more than 150,000 submissions across 75,000 individuals over 10 years and two natural experiments to rule out alternative explanations, we answer the question of how community members resolve this tension. We show that as their skill level increases, they become increasingly competitive and shift from using self-promotion to sabotaging their closest competitors. However, we also find signs of collaborative behavior when high-skilled members show leniency toward those community members who do not directly threaten their chance of winning. We explain how the individual-level use of strategic evaluations translates into important organizational-level outcomes by affecting the community structure through individuals' long-term participation. While low-skill targets of sabotage are less likely to participate in future contests, high-skill targets are more likely. This suggests a feedback loop between competitive evaluation behavior and future participation. These findings have important implications for the literature on crowdsourcing design, and the evolution and sustainability of crowdsourcing communities.