SentMix-3L: A Bangla-English-Hindi Code-Mixed Dataset for Sentiment Analysis

TL;DR

SentMix-3L is a new dataset for sentiment analysis involving Bangla, English, and Hindi code-mixed text, and demonstrates GPT-3.5's superior zero-shot performance over traditional models.

Contribution

Introduces SentMix-3L, the first multi-language code-mixed dataset for sentiment analysis involving three languages, and evaluates model performances on it.

Findings

GPT-3.5 zero-shot outperforms transformer-based models

SentMix-3L enables multi-language code-mixed sentiment analysis

Comprehensive evaluation of models on a novel three-language dataset

Abstract

Code-mixing is a well-studied linguistic phenomenon when two or more languages are mixed in text or speech. Several datasets have been build with the goal of training computational models for code-mixing. Although it is very common to observe code-mixing with multiple languages, most datasets available contain code-mixed between only two languages. In this paper, we introduce SentMix-3L, a novel dataset for sentiment analysis containing code-mixed data between three languages Bangla, English, and Hindi. We carry out a comprehensive evaluation using SentMix-3L. We show that zero-shot prompting with GPT-3.5 outperforms all transformer-based models on SentMix-3L.