Generating Effective Ensembles for Sentiment Analysis

TL;DR

This paper introduces a novel hierarchical ensemble construction (HEC) algorithm that combines transformer and traditional NLP models, significantly improving sentiment analysis accuracy over standard ensembles.

Contribution

The paper presents a new HEC algorithm for building mixed-model ensembles, enhancing sentiment analysis performance beyond existing methods.

Findings

Ensembles with mixed models outperform traditional ensembles.

Hierarchical construction via HEC improves ensemble effectiveness.

Proposed ensembles outperform GPT-4 on sentiment analysis tasks.

Abstract

In recent years, transformer models have revolutionized Natural Language Processing (NLP), achieving exceptional results across various tasks, including Sentiment Analysis (SA). As such, current state-of-the-art approaches for SA predominantly rely on transformer models alone, achieving impressive accuracy levels on benchmark datasets. In this paper, we show that the key for further improving the accuracy of such ensembles for SA is to include not only transformers, but also traditional NLP models, despite the inferiority of the latter compared to transformer models. However, as we empirically show, this necessitates a change in how the ensemble is constructed, specifically relying on the Hierarchical Ensemble Construction (HEC) algorithm we present. Our empirical studies across eight canonical SA datasets reveal that ensembles incorporating a mix of model types, structured via HEC, significantly outperform traditional ensembles. Finally, we provide a comparative analysis of the performance of the HEC and GPT-4, demonstrating that while GPT-4 closely approaches state-of-the-art SA methods, it remains outperformed by our proposed ensemble strategy.