Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews

TL;DR

This paper explores fine-tuning BERT models for Vietnamese sentiment analysis, demonstrating slight performance improvements over other embeddings and better results than original BERT fine-tuning methods.

Contribution

It introduces two BERT fine-tuning approaches for Vietnamese sentiment analysis and compares their effectiveness, showing improvements over previous methods.

Findings

BERT-based models outperform GloVe and FastText in Vietnamese sentiment analysis.

Using all BERT output vectors yields better performance than using only the [CLS] token.

The proposed BERT fine-tuning method surpasses the original BERT fine-tuning approach.

Abstract

Sentiment analysis is an important task in the field ofNature Language Processing (NLP), in which users' feedbackdata on a specific issue are evaluated and analyzed. Manydeep learning models have been proposed to tackle this task, including the recently-introduced Bidirectional Encoder Rep-resentations from Transformers (BERT) model. In this paper,we experiment with two BERT fine-tuning methods for thesentiment analysis task on datasets of Vietnamese reviews: 1) a method that uses only the [CLS] token as the input for anattached feed-forward neural network, and 2) another methodin which all BERT output vectors are used as the input forclassification. Experimental results on two datasets show thatmodels using BERT slightly outperform other models usingGloVe and FastText. Also, regarding the datasets employed inthis study, our proposed BERT fine-tuning method produces amodel with better performance than the original BERT fine-tuning method.