CineXDrama: Relevance Detection and Sentiment Analysis of Bangla YouTube Comments on Movie-Drama using Transformers: Insights from Interpretability Tool

TL;DR

This paper presents a system for filtering relevant Bangla YouTube comments and analyzing their sentiment using transformer models, with interpretability insights provided by LIME, based on a new annotated dataset.

Contribution

It introduces a relevance detection and sentiment analysis framework for Bangla comments using transformer models, and provides interpretability analysis with LIME.

Findings

BanglaBERT achieved 83.99% accuracy in relevance detection.

Sentiment analysis accuracy reached 93.3% with BanglaBERT.

The dataset contains 14,000 annotated comments.

Abstract

In recent years, YouTube has become the leading platform for Bangla movies and dramas, where viewers express their opinions in comments that convey their sentiments about the content. However, not all comments are relevant for sentiment analysis, necessitating a filtering mechanism. We propose a system that first assesses the relevance of comments and then analyzes the sentiment of those deemed relevant. We introduce a dataset of 14,000 manually collected and preprocessed comments, annotated for relevance (relevant or irrelevant) and sentiment (positive or negative). Eight transformer models, including BanglaBERT, were used for classification tasks, with BanglaBERT achieving the highest accuracy (83.99% for relevance detection and 93.3% for sentiment analysis). The study also integrates LIME to interpret model decisions, enhancing transparency.