BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews

TL;DR

This paper evaluates the effectiveness of fine-tuning BERT models, especially an Indonesian-specific pre-trained BERT, for sentiment analysis of Indonesian mobile app reviews, demonstrating superior accuracy over other models.

Contribution

It introduces the use of an Indonesian-specific BERT model for sentiment analysis and compares it with multilingual models, optimizing hyperparameters and labeling methods.

Findings

Indonesian BERT achieved up to 84% accuracy.

Indonesian BERT outperformed multilingual and traditional models.

Lexicon-based labeling improved model accuracy.

Abstract

User reviews have an essential role in the success of the developed mobile apps. User reviews in the textual form are unstructured data, creating a very high complexity when processed for sentiment analysis. Previous approaches that have been used often ignore the context of reviews. In addition, the relatively small data makes the model overfitting. A new approach, BERT, has been introduced as a transfer learning model with a pre-trained model that has previously been trained to have a better context representation. This study examines the effectiveness of fine-tuning BERT for sentiment analysis using two different pre-trained models. Besides the multilingual pre-trained model, we use the pre-trained model that only has been trained in Indonesian. The dataset used is Indonesian user reviews of the ten best apps in 2020 in Google Play sites. We also perform hyper-parameter tuning to find the optimum trained model. Two training data labeling approaches were also tested to determine the effectiveness of the model, which is score-based and lexicon-based. The experimental results show that pre-trained models trained in Indonesian have better average accuracy on lexicon-based data. The pre-trained Indonesian model highest accuracy is 84%, with 25 epochs and a training time of 24 minutes. These results are better than all of the machine learning and multilingual pre-trained models.