A Retail-Corpus for Aspect-Based Sentiment Analysis with Large Language Models

TL;DR

This paper introduces a new multilingual retail review dataset with aspect annotations and evaluates GPT-4 and LLaMA-3 on aspect-based sentiment analysis, showing GPT-4's superior performance.

Contribution

The study provides a large, manually annotated multilingual dataset for aspect-based sentiment analysis in retail and benchmarks state-of-the-art LLMs on this task.

Findings

GPT-4 achieves over 85% accuracy

LLaMA-3 performs slightly below GPT-4

Both models outperform previous baselines

Abstract

Aspect-based sentiment analysis enhances sentiment detection by associating it with specific aspects, offering deeper insights than traditional sentiment analysis. This study introduces a manually annotated dataset of 10,814 multilingual customer reviews covering brick-and-mortar retail stores, labeled with eight aspect categories and their sentiment. Using this dataset, the performance of GPT-4 and LLaMA-3 in aspect based sentiment analysis is evaluated to establish a baseline for the newly introduced data. The results show both models achieving over 85% accuracy, while GPT-4 outperforms LLaMA-3 overall with regard to all relevant metrics.