Self-Anchored Attention Model for Sample-Efficient Classification of Prosocial Text Chat

TL;DR

This paper introduces a novel Self-Anchored Attention Model (SAAM) that improves prosocial behavior classification in game chat by leveraging entire training sets as anchors, addressing low-resource challenges in online gaming environments.

Contribution

The paper presents the first automated system for classifying prosocial behaviors in in-game chat, utilizing a novel SAAM approach that enhances performance in low-resource settings.

Findings

SAAM achieves 7.9% improvement over existing methods.

Effective classification of prosocial behaviors in low-resource, real-world gaming data.

Application to Call of Duty demonstrates practical utility.

Abstract

Millions of players engage daily in competitive online games, communicating through in-game chat. Prior research has focused on detecting relatively small volumes of toxic content using various Natural Language Processing (NLP) techniques for the purpose of moderation. However, recent studies emphasize the importance of detecting prosocial communication, which can be as crucial as identifying toxic interactions. Recognizing prosocial behavior allows for its analysis, rewarding, and promotion. Unlike toxicity, there are limited datasets, models, and resources for identifying prosocial behaviors in game-chat text. In this work, we employed unsupervised discovery combined with game domain expert collaboration to identify and categorize prosocial player behaviors from game chat. We further propose a novel Self-Anchored Attention Model (SAAM) which gives 7.9% improvement compared to the best existing technique. The approach utilizes the entire training set as "anchors" to help improve model performance under the scarcity of training data. This approach led to the development of the first automated system for classifying prosocial behaviors in in-game chats, particularly given the low-resource settings where large-scale labeled data is not available. Our methodology was applied to one of the most popular online gaming titles - Call of Duty(R): Modern Warfare(R)II, showcasing its effectiveness. This research is novel in applying NLP techniques to discover and classify prosocial behaviors in player in-game chat communication. It can help shift the focus of moderation from solely penalizing toxicity to actively encouraging positive interactions on online platforms.