Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis

TL;DR

This paper investigates using ChatGPT as an automatic data annotation tool for sentiment analysis, demonstrating its superior accuracy over traditional lexicon-based methods across multiple datasets.

Contribution

It introduces ChatGPT as a novel, effective tool for sentiment data labeling, significantly improving annotation accuracy compared to existing lexicon-based approaches.

Findings

ChatGPT outperforms lexicon-based methods with 20-25% accuracy improvements.

It is effective across different sentiment analysis datasets.

ChatGPT can be used for diverse sentiment annotation tasks.

Abstract

Sentiment analysis is a well-known natural language processing task that involves identifying the emotional tone or polarity of a given piece of text. With the growth of social media and other online platforms, sentiment analysis has become increasingly crucial for businesses and organizations seeking to monitor and comprehend customer feedback as well as opinions. Supervised learning algorithms have been popularly employed for this task, but they require human-annotated text to create the classifier. To overcome this challenge, lexicon-based tools have been used. A drawback of lexicon-based algorithms is their reliance on pre-defined sentiment lexicons, which may not capture the full range of sentiments in natural language. ChatGPT is a new product of OpenAI and has emerged as the most popular AI product. It can answer questions on various topics and tasks. This study explores the use of ChatGPT as a tool for data labeling for different sentiment analysis tasks. It is evaluated on two distinct sentiment analysis datasets with varying purposes. The results demonstrate that ChatGPT outperforms other lexicon-based unsupervised methods with significant improvements in overall accuracy. Specifically, compared to the best-performing lexical-based algorithms, ChatGPT achieves a remarkable increase in accuracy of 20% for the tweets dataset and approximately 25% for the Amazon reviews dataset. These findings highlight the exceptional performance of ChatGPT in sentiment analysis tasks, surpassing existing lexicon-based approaches by a significant margin. The evidence suggests it can be used for annotation on different sentiment analysis events and taskss.