Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysis

TL;DR

This study evaluates zero- and few-shot prompting of large language models for Bangla sentiment analysis, comparing their performance to fine-tuned models using a new dataset of Bangla social media comments.

Contribution

It introduces a large annotated Bangla sentiment dataset and provides a comparative analysis of LLM prompting versus fine-tuning for low-resource language sentiment analysis.

Findings

Monolingual transformer models outperform other models in zero and few-shot settings.

LLMs like GPT-4 show promising results even without fine-tuning.

The dataset and tools will be publicly available for further research.

Abstract

The rapid expansion of the digital world has propelled sentiment analysis into a critical tool across diverse sectors such as marketing, politics, customer service, and healthcare. While there have been significant advancements in sentiment analysis for widely spoken languages, low-resource languages, such as Bangla, remain largely under-researched due to resource constraints. Furthermore, the recent unprecedented performance of Large Language Models (LLMs) in various applications highlights the need to evaluate them in the context of low-resource languages. In this study, we present a sizeable manually annotated dataset encompassing 33,606 Bangla news tweets and Facebook comments. We also investigate zero- and few-shot in-context learning with several language models, including Flan-T5, GPT-4, and Bloomz, offering a comparative analysis against fine-tuned models. Our findings suggest that monolingual transformer-based models consistently outperform other models, even in zero and few-shot scenarios. To foster continued exploration, we intend to make this dataset and our research tools publicly available to the broader research community.