Hope Speech Detection on Social Media Platforms

TL;DR

This paper explores machine learning techniques, including traditional and pre-trained models like BERT, to accurately identify hope speech in social media comments, emphasizing the importance of dataset relabeling for improved results.

Contribution

It introduces a relabeled dataset for hope speech detection and compares various machine learning models, highlighting the effectiveness of relabeling for better accuracy.

Findings

Relabeled dataset improves hope speech detection accuracy.

Pre-trained models outperform traditional machine learning methods.

Relabeling is crucial for dataset quality and model performance.

Abstract

Since personal computers became widely available in the consumer market, the amount of harmful content on the internet has significantly expanded. In simple terms, harmful content is anything online which causes a person distress or harm. It may include hate speech, violent content, threats, non-hope speech, etc. The online content must be positive, uplifting and supportive. Over the past few years, many studies have focused on solving this problem through hate speech detection, but very few focused on identifying hope speech. This paper discusses various machine learning approaches to identify a sentence as Hope Speech, Non-Hope Speech, or a Neutral sentence. The dataset used in the study contains English YouTube comments and is released as a part of the shared task "EACL-2021: Hope Speech Detection for Equality, Diversity, and Inclusion". Initially, the dataset obtained from the shared task had three classes: Hope Speech, non-Hope speech, and not in English; however, upon deeper inspection, we discovered that dataset relabeling is required. A group of undergraduates was hired to help perform the entire dataset's relabeling task. We experimented with conventional machine learning models (such as Na\"ive Bayes, logistic regression and support vector machine) and pre-trained models (such as BERT) on relabeled data. According to the experimental results, the relabeled data has achieved a better accuracy for Hope speech identification than the original data set.