CO-STAR: Conceptualisation of Stereotypes for Analysis and Reasoning

TL;DR

CO-STAR is a framework and dataset for identifying and analyzing implied stereotypes in language, addressing a subtle and understudied aspect of harmful content detection.

Contribution

The paper introduces CO-STAR, a novel conceptual framework and a large annotated dataset for implied stereotypes, advancing research in subtle hate speech detection.

Findings

Achieved state-of-the-art results on stereotype detection

Created a dataset with over 12,000 annotations

Identified limitations in understanding complex stereotypes

Abstract

Warning: this paper contains material which may be offensive or upsetting. While much of recent work has focused on the detection of hate speech and overtly offensive content, very little research has explored the more subtle but equally harmful language in the form of implied stereotypes. This is a challenging domain, made even more so by the fact that humans often struggle to understand and reason about stereotypes. We build on existing literature and present CO-STAR (COnceptualisation of STereotypes for Analysis and Reasoning), a novel framework which encodes the underlying concepts of implied stereotypes. We also introduce the CO-STAR training data set, which contains just over 12K structured annotations of implied stereotypes and stereotype conceptualisations, and achieve state-of-the-art results after training and manual evaluation. The CO-STAR models are, however, limited in their ability to understand more complex and subtly worded stereotypes, and our research motivates future work in developing models with more sophisticated methods for encoding common-sense knowledge.