Improving Document-Level Sentiment Analysis with User and Product Context

TL;DR

This paper enhances document-level sentiment analysis by incorporating historical review texts from users and products, using a non-hierarchical model that directly attends to words, leading to significant performance improvements.

Contribution

It introduces a method to include all available historical reviews for users and products, and removes hierarchical architecture to improve attention mechanisms.

Findings

Achieved over 2% improvement on IMDB, Yelp 2013, and Yelp 2014 datasets.

Demonstrated the effectiveness of incorporating historical review context.

Showed that direct word-to-word attention benefits sentiment prediction.

Abstract

Past work that improves document-level sentiment analysis by encoding user and product information has been limited to considering only the text of the current review. We investigate incorporating additional review text available at the time of sentiment prediction that may prove meaningful for guiding prediction. Firstly, we incorporate all available historical review text belonging to the author of the review in question. Secondly, we investigate the inclusion of historical reviews associated with the current product (written by other users). We achieve this by explicitly storing representations of reviews written by the same user and about the same product and force the model to memorize all reviews for one particular user and product. Additionally, we drop the hierarchical architecture used in previous work to enable words in the text to directly attend to each other. Experiment results on IMDB, Yelp 2013 and Yelp 2014 datasets show improvement to state-of-the-art of more than 2 percentage points in the best case.