Bridging the gap in online hate speech detection: a comparative analysis of BERT and traditional models for homophobic content identification on X/Twitter

TL;DR

This study compares BERT and traditional models for detecting homophobic content on X/Twitter, highlighting BERT's superior performance and emphasizing the importance of validation techniques, while releasing a large open-source dataset.

Contribution

It introduces the largest open-source dataset for homophobia detection and provides a comparative analysis of BERT and traditional models, advancing hate speech detection research.

Findings

BERT outperforms traditional models in detecting homophobic content.

Validation technique choice significantly affects model performance.

The released dataset supports future research in hate speech detection.

Abstract

Our study addresses a significant gap in online hate speech detection research by focusing on homophobia, an area often neglected in sentiment analysis research. Utilising advanced sentiment analysis models, particularly BERT, and traditional machine learning methods, we developed a nuanced approach to identify homophobic content on X/Twitter. This research is pivotal due to the persistent underrepresentation of homophobia in detection models. Our findings reveal that while BERT outperforms traditional methods, the choice of validation technique can impact model performance. This underscores the importance of contextual understanding in detecting nuanced hate speech. By releasing the largest open-source labelled English dataset for homophobia detection known to us, an analysis of various models' performance and our strongest BERT-based model, we aim to enhance online safety and inclusivity. Future work will extend to broader LGBTQIA+ hate speech detection, addressing the challenges of sourcing diverse datasets. Through this endeavour, we contribute to the larger effort against online hate, advocating for a more inclusive digital landscape. Our study not only offers insights into the effective detection of homophobic content by improving on previous research results, but it also lays groundwork for future advancements in hate speech analysis.