Towards Detecting Contextual Real-Time Toxicity for In-Game Chat

TL;DR

This paper presents ToxBuster, a scalable real-time toxicity detection model that leverages chat history and metadata, outperforming existing models in multiplayer gaming environments and aiding moderation efforts.

Contribution

The paper introduces ToxBuster, a novel real-time toxicity detection model that incorporates chat context and metadata, demonstrating superior performance and transferability across multiple games.

Findings

ToxBuster outperforms conventional models in multiplayer games.

Successfully flags 82.1% of toxic players with high precision.

Proactively moderates an additional 6% of unreported toxic players.

Abstract

Real-time toxicity detection in online environments poses a significant challenge, due to the increasing prevalence of social media and gaming platforms. We introduce ToxBuster, a simple and scalable model that reliably detects toxic content in real-time for a line of chat by including chat history and metadata. ToxBuster consistently outperforms conventional toxicity models across popular multiplayer games, including Rainbow Six Siege, For Honor, and DOTA 2. We conduct an ablation study to assess the importance of each model component and explore ToxBuster's transferability across the datasets. Furthermore, we showcase ToxBuster's efficacy in post-game moderation, successfully flagging 82.1% of chat-reported players at a precision level of 90.0%. Additionally, we show how an additional 6% of unreported toxic players can be proactively moderated.