Ensemble BERT: A student social network text sentiment classification model based on ensemble learning and BERT architecture

TL;DR

This paper proposes an ensemble learning approach using multiple BERT models to classify emotional tendencies in middle school students' social network texts, improving performance while considering efficiency and interpretability.

Contribution

It introduces a novel ensemble BERT-based network for social network text sentiment classification, balancing accuracy, training time, and interpretability.

Findings

Ensemble BERT outperforms individual models in sentiment classification.

Ensemble of three single-layer BERT models is comparable to a three-layer BERT model.

Deeper BERT networks are more efficient for training while maintaining performance.

Abstract

The mental health assessment of middle school students has always been one of the focuses in the field of education. This paper introduces a new ensemble learning network based on BERT, employing the concept of enhancing model performance by integrating multiple classifiers. We trained a range of BERT-based learners, which combined using the majority voting method. We collect social network text data of middle school students through China's Weibo and apply the method to the task of classifying emotional tendencies in middle school students' social network texts. Experimental results suggest that the ensemble learning network has a better performance than the base model and the performance of the ensemble learning model, consisting of three single-layer BERT models, is barely the same as a three-layer BERT model but requires 11.58% more training time. Therefore, in terms of balancing prediction effect and efficiency, the deeper BERT network should be preferred for training. However, for interpretability, network ensembles can provide acceptable solutions.