Dataset for Identification of Homophobia and Transophobia in Multilingual YouTube Comments

TL;DR

This paper introduces a new multilingual dataset with expert annotations for identifying homophobic and transphobic comments on YouTube, along with baseline models, to aid automatic detection of hate speech against LGBT+ communities.

Contribution

It provides the first expert-labelled, multilingual dataset for online homophobia and transphobia detection, including a hierarchical taxonomy and baseline models.

Findings

15,141 annotated comments in the dataset

High inter-annotator agreement achieved

Baseline models demonstrate effective initial detection performance

Abstract

The increased proliferation of abusive content on social media platforms has a negative impact on online users. The dread, dislike, discomfort, or mistrust of lesbian, gay, transgender or bisexual persons is defined as homophobia/transphobia. Homophobic/transphobic speech is a type of offensive language that may be summarized as hate speech directed toward LGBT+ people, and it has been a growing concern in recent years. Online homophobia/transphobia is a severe societal problem that can make online platforms poisonous and unwelcome to LGBT+ people while also attempting to eliminate equality, diversity, and inclusion. We provide a new hierarchical taxonomy for online homophobia and transphobia, as well as an expert-labelled dataset that will allow homophobic/transphobic content to be automatically identified. We educated annotators and supplied them with comprehensive annotation rules because this is a sensitive issue, and we previously discovered that untrained crowdsourcing annotators struggle with diagnosing homophobia due to cultural and other prejudices. The dataset comprises 15,141 annotated multilingual comments. This paper describes the process of building the dataset, qualitative analysis of data, and inter-annotator agreement. In addition, we create baseline models for the dataset. To the best of our knowledge, our dataset is the first such dataset created. Warning: This paper contains explicit statements of homophobia, transphobia, stereotypes which may be distressing to some readers.