STFU NOOB! Predicting Crowdsourced Decisions on Toxic Behavior in Online Games
Jeremy Blackburn, Haewoon Kwak

TL;DR
This paper presents a supervised learning model that predicts crowdsourced decisions on toxic behavior in online games, aiming to reduce costs and improve efficiency in moderation systems like League of Legends.
Contribution
It introduces a large-scale dataset and a predictive model for crowdsourced toxic behavior decisions, demonstrating high accuracy and regional portability.
Findings
Effective detection of majority-vote toxic cases
High accuracy in predicting crowdsourced decisions
Potential for significant cost savings in moderation
Abstract
One problem facing players of competitive games is negative, or toxic, behavior. League of Legends, the largest eSport game, uses a crowdsourcing platform called the Tribunal to judge whether a reported toxic player should be punished or not. The Tribunal is a two stage system requiring reports from those players that directly observe toxic behavior, and human experts that review aggregated reports. While this system has successfully dealt with the vague nature of toxic behavior by majority rules based on many votes, it naturally requires tremendous cost, time, and human efforts. In this paper, we propose a supervised learning approach for predicting crowdsourced decisions on toxic behavior with large-scale labeled data collections; over 10 million user reports involved in 1.46 million toxic players and corresponding crowdsourced decisions. Our result shows good performance in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMobile Crowdsensing and Crowdsourcing · Open Source Software Innovations · Digital Marketing and Social Media
