Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification

TL;DR

This paper improves aspect-target sentiment classification by domain-specific BERT finetuning, achieving state-of-the-art results and demonstrating robustness across domains, with insights into model errors.

Contribution

It introduces a two-step domain adaptation approach for BERT in ATSC, significantly enhancing performance and robustness over baseline models.

Findings

State-of-the-art results on SemEval 2014 dataset

Cross-domain BERT finetuning outperforms vanilla BERT and XLNet

Model error analysis provides interpretability insights.

Abstract

Aspect-Target Sentiment Classification (ATSC) is a subtask of Aspect-Based Sentiment Analysis (ABSA), which has many applications e.g. in e-commerce, where data and insights from reviews can be leveraged to create value for businesses and customers. Recently, deep transfer-learning methods have been applied successfully to a myriad of Natural Language Processing (NLP) tasks, including ATSC. Building on top of the prominent BERT language model, we approach ATSC using a two-step procedure: self-supervised domain-specific BERT language model finetuning, followed by supervised task-specific finetuning. Our findings on how to best exploit domain-specific language model finetuning enable us to produce new state-of-the-art performance on the SemEval 2014 Task 4 restaurants dataset. In addition, to explore the real-world robustness of our models, we perform cross-domain evaluation. We show that a cross-domain adapted BERT language model performs significantly better than strong baseline models like vanilla BERT-base and XLNet-base. Finally, we conduct a case study to interpret model prediction errors.