TEDB System Description to a Shared Task on Euphemism Detection 2022
Peratham Wiriyathammabhum

TL;DR
This paper presents a Transformer-based system for euphemism detection, achieving an 81.6% F1-score by fine-tuning RoBERTa and KimCNN, exploring various training schemes and pretrained models.
Contribution
It introduces a novel baseline for euphemism detection using Transformer models and evaluates different training strategies and pretrained models for improved performance.
Findings
Pretrained sentiment and offensiveness models improve F1-score.
Pretraining on sarcasm detection yields lower F1-scores.
Adding more word vector channels does not enhance performance.
Abstract
In this report, we describe our Transformers for euphemism detection baseline (TEDB) submissions to a shared task on euphemism detection 2022. We cast the task of predicting euphemism as text classification. We considered Transformer-based models which are the current state-of-the-art methods for text classification. We explored different training schemes, pretrained models, and model architectures. Our best result of 0.816 F1-score (0.818 precision and 0.814 recall) consists of a euphemism-detection-finetuned TweetEval/TimeLMs-pretrained RoBERTa model as a feature extractor frontend with a KimCNN classifier backend trained end-to-end using a cosine annealing scheduler. We observed pretrained models on sentiment analysis and offensiveness detection to correlate with more F1-score while pretraining on other tasks, such as sarcasm detection, produces less F1-scores. Also, putting more…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHate Speech and Cyberbullying Detection · Swearing, Euphemism, Multilingualism · Authorship Attribution and Profiling
MethodsMulti-Head Attention · Attention Is All You Need · Linear Layer · Refunds@Expedia|||How do I get a full refund from Expedia? · Linear Warmup With Linear Decay · Attention Dropout · Weight Decay · Residual Connection · Dense Connections · Layer Normalization
