Reducing Computational Costs in Sentiment Analysis: Tensorized Recurrent Networks vs. Recurrent Networks

TL;DR

This paper compares traditional recurrent neural networks with tensorized versions for sentiment analysis, demonstrating that tensorized models achieve similar accuracy while significantly reducing computational resource requirements.

Contribution

The study introduces tensorized recurrent networks for sentiment analysis and shows they perform comparably to traditional models with fewer parameters and less computational cost.

Findings

Tensorized models reach similar accuracy as traditional recurrent networks.

Tensorized models require fewer parameters and less training resources.

Comparable performance achieved with reduced computational costs.

Abstract

Anticipating audience reaction towards a certain text is integral to several facets of society ranging from politics, research, and commercial industries. Sentiment analysis (SA) is a useful natural language processing (NLP) technique that utilizes lexical/statistical and deep learning methods to determine whether different-sized texts exhibit positive, negative, or neutral emotions. Recurrent networks are widely used in machine-learning communities for problems with sequential data. However, a drawback of models based on Long-Short Term Memory networks and Gated Recurrent Units is the significantly high number of parameters, and thus, such models are computationally expensive. This drawback is even more significant when the available data are limited. Also, such models require significant over-parameterization and regularization to achieve optimal performance. Tensorized models represent a potential solution. In this paper, we classify the sentiment of some social media posts. We compare traditional recurrent models with their tensorized version, and we show that with the tensorized models, we reach comparable performances with respect to the traditional models while using fewer resources for the training.