Emotion Detection with Transformers: A Comparative Study

TL;DR

This paper compares various transformer models for emotion detection in text, analyzing how different preprocessing and fine-tuning strategies affect their performance, revealing that some common text cleaning methods may reduce accuracy.

Contribution

It provides a comprehensive comparison of transformer-based models for emotion classification and investigates the impact of preprocessing and fine-tuning choices on their effectiveness.

Findings

Removing punctuation and stop words can hinder model performance.

Transformers excel at understanding contextual relationships in text.

Fine-tuning strategies significantly influence emotion detection accuracy.

Abstract

In this study, we explore the application of transformer-based models for emotion classification on text data. We train and evaluate several pre-trained transformer models, on the Emotion dataset using different variants of transformers. The paper also analyzes some factors that in-fluence the performance of the model, such as the fine-tuning of the transformer layer, the trainability of the layer, and the preprocessing of the text data. Our analysis reveals that commonly applied techniques like removing punctuation and stop words can hinder model performance. This might be because transformers strength lies in understanding contextual relationships within text. Elements like punctuation and stop words can still convey sentiment or emphasis and removing them might disrupt this context.