SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning

TL;DR

This paper evaluates GPT-based methods for sentiment analysis on SemEval 2017, demonstrating their superior performance over existing models and exploring strategies like prompt engineering and fine-tuning.

Contribution

It introduces and compares three GPT-based sentiment analysis strategies, highlighting their effectiveness and potential advantages over traditional models.

Findings

GPT approaches outperform state-of-the-art by over 22% in F1-score

GPT models better handle context and sarcasm in sentiment tasks

Prompt engineering and fine-tuning enhance GPT performance

Abstract

This study presents a thorough examination of various Generative Pretrained Transformer (GPT) methodologies in sentiment analysis, specifically in the context of Task 4 on the SemEval 2017 dataset. Three primary strategies are employed: 1) prompt engineering using the advanced GPT-3.5 Turbo, 2) fine-tuning GPT models, and 3) an inventive approach to embedding classification. The research yields detailed comparative insights among these strategies and individual GPT models, revealing their unique strengths and potential limitations. Additionally, the study compares these GPT-based methodologies with other current, high-performing models previously used with the same dataset. The results illustrate the significant superiority of the GPT approaches in terms of predictive performance, more than 22\% in F1-score compared to the state-of-the-art. Further, the paper sheds light on common challenges in sentiment analysis tasks, such as understanding context and detecting sarcasm. It underscores the enhanced capabilities of the GPT models to effectively handle these complexities. Taken together, these findings highlight the promising potential of GPT models in sentiment analysis, setting the stage for future research in this field. The code can be found at https://github.com/DSAatUSU/SentimentGPT