An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction

TL;DR

This paper empirically investigates how position embeddings and various text encoders, including GCNs, impact target-oriented opinion words extraction, revealing that simpler models can outperform complex neural structures.

Contribution

It systematically evaluates the contribution of position embeddings and different encoders, demonstrating the effectiveness of simple BiLSTM models over complex architectures.

Findings

BiLSTM models effectively encode position information.

Graph convolutional networks provide marginal improvements.

Simple models outperform several complex neural structures.

Abstract

Target-oriented opinion words extraction (TOWE) (Fan et al., 2019b) is a new subtask of target-oriented sentiment analysis that aims to extract opinion words for a given aspect in text. Current state-of-the-art methods leverage position embeddings to capture the relative position of a word to the target. However, the performance of these methods depends on the ability to incorporate this information into word representations. In this paper, we explore a variety of text encoders based on pretrained word embeddings or language models that leverage part-of-speech and position embeddings, aiming to examine the actual contribution of each component in TOWE. We also adapt a graph convolutional network (GCN) to enhance word representations by incorporating syntactic information. Our experimental results demonstrate that BiLSTM-based models can effectively encode position information into word representations while using a GCN only achieves marginal gains. Interestingly, our simple methods outperform several state-of-the-art complex neural structures.