Attentional Encoder Network for Targeted Sentiment Classification

TL;DR

This paper introduces an Attentional Encoder Network that replaces RNNs with attention mechanisms for targeted sentiment classification, improving efficiency and performance, especially when combined with BERT.

Contribution

The paper proposes a novel attention-based encoder architecture that avoids recurrence, addresses label unreliability with smoothing, and achieves state-of-the-art results with BERT.

Findings

The AEN model outperforms previous RNN-based methods.

Label smoothing improves classification robustness.

Combining AEN with BERT yields top performance.

Abstract

Targeted sentiment classification aims at determining the sentimental tendency towards specific targets. Most of the previous approaches model context and target words with RNN and attention. However, RNNs are difficult to parallelize and truncated backpropagation through time brings difficulty in remembering long-term patterns. To address this issue, this paper proposes an Attentional Encoder Network (AEN) which eschews recurrence and employs attention based encoders for the modeling between context and target. We raise the label unreliability issue and introduce label smoothing regularization. We also apply pre-trained BERT to this task and obtain new state-of-the-art results. Experiments and analysis demonstrate the effectiveness and lightweight of our model.