Multiclass Hate Speech Detection with RoBERTa-OTA: Integrating Transformer Attention and Graph Convolutional Networks

TL;DR

This paper introduces RoBERTa-OTA, a novel model combining transformer attention and graph neural networks with ontological knowledge to improve multiclass hate speech detection across demographic categories, achieving higher accuracy and efficiency.

Contribution

The paper presents a new architecture that integrates structured domain knowledge into transformer-based hate speech detection models, enhancing classification performance and interpretability.

Findings

Achieves 96.04% accuracy on balanced social media samples.

Improves gender-based hate speech detection by 2.36 percentage points.

Maintains computational efficiency with minimal parameter overhead.

Abstract

Multiclass hate speech detection across demographic categories remains computationally challenging due to implicit targeting strategies and linguistic variability in social media content. Existing approaches rely solely on learned representations from training data, without explicitly incorporating structured ontological frameworks that can enhance classification through formal domain knowledge integration. We propose RoBERTa-OTA, which introduces ontology-guided attention mechanisms that process textual features alongside structured knowledge representations through enhanced Graph Convolutional Networks. The architecture combines RoBERTa embeddings with scaled attention layers and graph neural networks to integrate contextual language understanding with domain-specific semantic knowledge. Evaluation across 39,747 balanced samples using 5-fold cross-validation demonstrates significant performance gains over baseline RoBERTa implementations and existing state-of-the-art methods. RoBERTa-OTA achieves 96.04\% accuracy compared to 95.02\% for standard RoBERTa, with substantial improvements for challenging categories: gender-based hate speech detection improves by 2.36 percentage points while other hate speech categories improve by 2.38 percentage points. The enhanced architecture maintains computational efficiency with only 0.33\% parameter overhead, providing practical advantages for large-scale content moderation applications requiring fine-grained demographic hate speech classification.