Fine-Grained Emotion Detection on GoEmotions: Experimental Comparison of Classical Machine Learning, BiLSTM, and Transformer Models

TL;DR

This paper compares classical ML, BiLSTM, and BERT models for fine-grained emotion detection on GoEmotions, highlighting the strengths of each approach in handling label imbalance and overlap.

Contribution

It provides a comprehensive benchmark of three modeling approaches on GoEmotions, demonstrating the effectiveness of BERT in balancing multiple evaluation metrics.

Findings

Logistic regression achieved highest Micro-F1 of 0.51.

BERT outperformed others in Macro-F1, Hamming Loss, and Subset Accuracy.

Frequent emotions are surface-level, while contextual models better detect rare and ambiguous emotions.

Abstract

Fine-grained emotion recognition is a challenging multi-label NLP task due to label overlap and class imbalance. In this work, we benchmark three modeling families on the GoEmotions dataset: a TF-IDF-based logistic regression system trained with binary relevance, a BiLSTM with attention, and a BERT model fine-tuned for multi-label classification. Experiments follow the official train/validation/test split, and imbalance is mitigated using inverse-frequency class weights. Across several metrics, namely Micro-F1, Macro-F1, Hamming Loss, and Subset Accuracy, we observe that logistic regression attains the highest Micro-F1 of 0.51, while BERT achieves the best overall balance surpassing the official paper's reported results, reaching Macro-F1 0.49, Hamming Loss 0.036, and Subset Accuracy 0.36. This suggests that frequent emotions often rely on surface lexical cues, whereas contextual representations improve performance on rarer emotions and more ambiguous examples.