Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis

TL;DR

This paper enhances Indonesian aspect-based sentiment analysis by fine-tuning multilingual BERT, significantly improving accuracy over previous CNN and XGBoost models, especially in handling out-of-vocabulary words.

Contribution

It introduces a method combining multilingual BERT with task transformation for Indonesian ABSA, achieving an 8% F1-score improvement over prior models.

Findings

8% F1-score improvement over previous models

Better generalization to test data

Effective handling of OOV words

Abstract

Although previous research on Aspect-based Sentiment Analysis (ABSA) for Indonesian reviews in hotel domain has been conducted using CNN and XGBoost, its model did not generalize well in test data and high number of OOV words contributed to misclassification cases. Nowadays, most state-of-the-art results for wide array of NLP tasks are achieved by utilizing pretrained language representation. In this paper, we intend to incorporate one of the foremost language representation model, BERT, to perform ABSA in Indonesian reviews dataset. By combining multilingual BERT (m-BERT) with task transformation method, we manage to achieve significant improvement by 8% on the F1-score compared to the result from our previous study.