Efficient Sentiment Analysis: A Resource-Aware Evaluation of Feature Extraction Techniques, Ensembling, and Deep Learning Models

TL;DR

This paper evaluates various sentiment analysis models focusing on resource efficiency, showing that some configurations offer significant resource savings with minimal accuracy loss, especially on smaller datasets.

Contribution

It provides a comprehensive resource-aware comparison of sentiment analysis techniques, including feature extraction, ensembling, deep learning, and large language models.

Findings

Fine-tuned LLM achieves highest accuracy.

Certain configurations save up to 24-283 times resources with less than 1% accuracy loss.

Resource differences are more pronounced on smaller datasets.

Abstract

While reaching for NLP systems that maximize accuracy, other important metrics of system performance are often overlooked. Prior models are easily forgotten despite their possible suitability in settings where large computing resources are unavailable or relatively more costly. In this paper, we perform a broad comparative evaluation of document-level sentiment analysis models with a focus on resource costs that are important for the feasibility of model deployment and general climate consciousness. Our experiments consider different feature extraction techniques, the effect of ensembling, task-specific deep learning modeling, and domain-independent large language models (LLMs). We find that while a fine-tuned LLM achieves the best accuracy, some alternate configurations provide huge (up to 24, 283 *) resource savings for a marginal (<1%) loss in accuracy. Furthermore, we find that for smaller datasets, the differences in accuracy shrink while the difference in resource consumption grows further.