TEDB System Description to a Shared Task on Euphemism Detection 2022

TL;DR

This paper presents a Transformer-based system for euphemism detection, achieving an 81.6% F1-score by fine-tuning RoBERTa and KimCNN, exploring various training schemes and pretrained models.

Contribution

It introduces a novel baseline for euphemism detection using Transformer models and evaluates different training strategies and pretrained models for improved performance.

Findings

Pretrained sentiment and offensiveness models improve F1-score.

Pretraining on sarcasm detection yields lower F1-scores.

Adding more word vector channels does not enhance performance.

Abstract

In this report, we describe our Transformers for euphemism detection baseline (TEDB) submissions to a shared task on euphemism detection 2022. We cast the task of predicting euphemism as text classification. We considered Transformer-based models which are the current state-of-the-art methods for text classification. We explored different training schemes, pretrained models, and model architectures. Our best result of 0.816 F1-score (0.818 precision and 0.814 recall) consists of a euphemism-detection-finetuned TweetEval/TimeLMs-pretrained RoBERTa model as a feature extractor frontend with a KimCNN classifier backend trained end-to-end using a cosine annealing scheduler. We observed pretrained models on sentiment analysis and offensiveness detection to correlate with more F1-score while pretraining on other tasks, such as sarcasm detection, produces less F1-scores. Also, putting more word vector channels does not improve the performance in our experiments.