Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis

TL;DR

This study evaluates various machine learning and NLP models, including transformer-based ones, for sentiment analysis on Reddit comments using the GoEmotions dataset, highlighting RoBERTa's superior performance.

Contribution

It expands prior work by assessing a diverse set of models and evaluation criteria, including hierarchical classification and efficiency, on a large emotion-labeled Reddit comment dataset.

Findings

RoBERTa outperforms traditional classifiers in accuracy

Hierarchical classification improves emotion detection granularity

Computational efficiency varies across models

Abstract

Sentiment analysis, an increasingly vital field in both academia and industry, plays a pivotal role in machine learning applications, particularly on social media platforms like Reddit. However, the efficacy of sentiment analysis models is hindered by the lack of expansive and fine-grained emotion datasets. To address this gap, our study leverages the GoEmotions dataset, comprising a diverse range of emotions, to evaluate sentiment analysis methods across a substantial corpus of 58,000 comments. Distinguished from prior studies by the Google team, which limited their analysis to only two models, our research expands the scope by evaluating a diverse array of models. We investigate the performance of traditional classifiers such as Naive Bayes and Support Vector Machines (SVM), as well as state-of-the-art transformer-based models including BERT, RoBERTa, and GPT. Furthermore, our evaluation criteria extend beyond accuracy to encompass nuanced assessments, including hierarchical classification based on varying levels of granularity in emotion categorization. Additionally, considerations such as computational efficiency are incorporated to provide a comprehensive evaluation framework. Our findings reveal that the RoBERTa model consistently outperforms the baseline models, demonstrating superior accuracy in fine-grained sentiment classification tasks. This underscores the substantial potential and significance of the RoBERTa model in advancing sentiment analysis capabilities.