Parsimonious Argument Annotations for Hate Speech Counter-narratives

TL;DR

This paper enhances the Hateval corpus with argumentative annotations to improve automated hate speech counter-narrative generation, showing promising results for some argument components but also highlighting annotation challenges.

Contribution

It introduces argumentative annotations based on Wagemanns' framework to the Hateval corpus, aiding in developing more effective counter-narratives for hate speech.

Findings

Automatic annotators perform comparably to humans on some argument aspects.

Certain argumentative elements have low inter-annotator agreement.

Preliminary results suggest potential for improved counter-narrative generation.

Abstract

We present an enrichment of the Hateval corpus of hate speech tweets (Basile et. al 2019) aimed to facilitate automated counter-narrative generation. Comparably to previous work (Chung et. al. 2019), manually written counter-narratives are associated to tweets. However, this information alone seems insufficient to obtain satisfactory language models for counter-narrative generation. That is why we have also annotated tweets with argumentative information based on Wagemanns (2016), that we believe can help in building convincing and effective counter-narratives for hate speech against particular groups. We discuss adequacies and difficulties of this annotation process and present several baselines for automatic detection of the annotated elements. Preliminary results show that automatic annotators perform close to human annotators to detect some aspects of argumentation, while others only reach low or moderate level of inter-annotator agreement.