Detecting Community Sensitive Norm Violations in Online Conversations

TL;DR

This paper presents a new dataset and models for detecting a broader range of community norm violations in online conversations, emphasizing context and community sensitivity to improve moderation accuracy.

Contribution

It introduces a comprehensive dataset and models that detect diverse norm violations considering local and global community contexts, advancing beyond toxicity detection.

Findings

Models achieve high performance in detecting norm violations.

Context-aware approaches outperform baseline toxicity detection methods.

Dataset covers a wider spectrum of community norms.

Abstract

Online platforms and communities establish their own norms that govern what behavior is acceptable within the community. Substantial effort in NLP has focused on identifying unacceptable behaviors and, recently, on forecasting them before they occur. However, these efforts have largely focused on toxicity as the sole form of community norm violation. Such focus has overlooked the much larger set of rules that moderators enforce. Here, we introduce a new dataset focusing on a more complete spectrum of community norms and their violations in the local conversational and global community contexts. We introduce a series of models that use this data to develop context- and community-sensitive norm violation detection, showing that these changes give high performance.