Transformer-Encoder-GRU (T-E-GRU) for Chinese Sentiment Analysis on Chinese Comment Text

TL;DR

This paper introduces T-E-GRU, a hybrid model combining transformer encoder and GRU, to improve Chinese sentiment analysis by better capturing sequence features and handling punctuation issues, outperforming traditional models.

Contribution

The paper proposes a novel T-E-GRU model that integrates transformer encoder and GRU for Chinese sentiment analysis, addressing limitations of position encoding and punctuation handling.

Findings

T-E-GRU outperforms classic recurrent models.

Selective punctuation retention improves segmentation.

Experimental results on three datasets confirm effectiveness.

Abstract

Chinese sentiment analysis (CSA) has always been one of the challenges in natural language processing due to its complexity and uncertainty. Transformer has succeeded in capturing semantic features, but it uses position encoding to capture sequence features, which has great shortcomings compared with the recurrent model. In this paper, we propose T-E-GRU for Chinese sentiment analysis, which combine transformer encoder and GRU. We conducted experiments on three Chinese comment datasets. In view of the confusion of punctuation marks in Chinese comment texts, we selectively retain some punctuation marks with sentence segmentation ability. The experimental results show that T-E-GRU outperforms classic recurrent model and recurrent model with attention.