Revisiting the Role of Label Smoothing in Enhanced Text Sentiment Classification

TL;DR

This paper provides an in-depth analysis of how label smoothing improves text sentiment classification across multiple datasets and models, highlighting its benefits in accelerating convergence and enhancing sample distinguishability.

Contribution

It offers a detailed examination of label smoothing's effects on sentiment classification, including parameter tuning and its impact on model convergence and sample separability.

Findings

Label smoothing improves performance on most datasets.

It accelerates model convergence.

It makes samples of different labels more distinguishable.

Abstract

Label smoothing is a widely used technique in various domains, such as text classification, image classification and speech recognition, known for effectively combating model overfitting. However, there is little fine-grained analysis on how label smoothing enhances text sentiment classification. To fill in the gap, this article performs a set of in-depth analyses on eight datasets for text sentiment classification and three deep learning architectures: TextCNN, BERT, and RoBERTa, under two learning schemes: training from scratch and fine-tuning. By tuning the smoothing parameters, we can achieve improved performance on almost all datasets for each model architecture. We further investigate the benefits of label smoothing, finding that label smoothing can accelerate the convergence of deep models and make samples of different labels easily distinguishable.