Sentiment Classification of Customer Reviews about Automobiles in Roman Urdu

TL;DR

This paper presents a sentiment classification approach for Roman Urdu automobile reviews using WEKA, demonstrating that Multinomial Naive Bayes outperforms other models in accuracy and other metrics.

Contribution

It introduces a novel sentiment analysis framework for Roman Urdu reviews, applying multiple classifiers and identifying the most effective model for this language and domain.

Findings

Multinomial Naive Bayes achieved the highest accuracy.

Roman Urdu sentiment classification is feasible with machine learning.

The dataset contained 2000 reviews, balanced between positive and negative.

Abstract

Text mining is a broad field having sentiment mining as its important constituent in which we try to deduce the behavior of people towards a specific item, merchandise, politics, sports, social media comments, review sites etc. Out of many issues in sentiment mining, analysis and classification, one major issue is that the reviews and comments can be in different languages like English, Arabic, Urdu etc. Handling each language according to its rules is a difficult task. A lot of research work has been done in English Language for sentiment analysis and classification but limited sentiment analysis work is being carried out on other regional languages like Arabic, Urdu and Hindi. In this paper, Waikato Environment for Knowledge Analysis (WEKA) is used as a platform to execute different classification models for text classification of Roman Urdu text. Reviews dataset has been scrapped from different automobiles sites. These extracted Roman Urdu reviews, containing 1000 positive and 1000 negative reviews, are then saved in WEKA attribute-relation file format (arff) as labeled examples. Training is done on 80% of this data and rest of it is used for testing purpose which is done using different models and results are analyzed in each case. The results show that Multinomial Naive Bayes outperformed Bagging, Deep Neural Network, Decision Tree, Random Forest, AdaBoost, k-NN and SVM Classifiers in terms of more accuracy, precision, recall and F-measure.