Studying Socially Unacceptable Discourse Classification (SUD) through different eyes: "Are we on the same page ?"

TL;DR

This paper introduces a new annotated corpus for Socially Unacceptable Discourse detection, evaluates classifier generalization across contexts, and discusses challenges and insights for improving SUD classification.

Contribution

It presents a novel, diverse corpus for SUD detection and analyzes how annotation differences affect classifier performance across online sources.

Findings

The corpus covers various online sources and SUD categories.

Classifier generalization varies across different annotation modalities.

Data insights support improved annotation and detection strategies.

Abstract

We study Socially Unacceptable Discourse (SUD) characterization and detection in online text. We first build and present a novel corpus that contains a large variety of manually annotated texts from different online sources used so far in state-of-the-art Machine learning (ML) SUD detection solutions. This global context allows us to test the generalization ability of SUD classifiers that acquire knowledge around the same SUD categories, but from different contexts. From this perspective, we can analyze how (possibly) different annotation modalities influence SUD learning by discussing open challenges and open research directions. We also provide several data insights which can support domain experts in the annotation task.