UoT-UWF-PartAI at SemEval-2021 Task 5: Self Attention Based Bi-GRU with Multi-Embedding Representation for Toxicity Highlighter

TL;DR

This paper introduces a token-level toxicity detection model using self-attention and multi-embedding representations, combining GPT-2, GloVe, and RoBERTa embeddings, achieving promising results in highlighting toxic spans.

Contribution

It presents a novel token-level toxicity detection model based on self-attention and multi-embedding representations, which improves span detection accuracy.

Findings

Effective detection of toxic spans demonstrated

Multi-embedding approach enhances token representation

Model outperforms previous token-level toxicity methods

Abstract

Toxic Spans Detection(TSD) task is defined as highlighting spans that make a text toxic. Many works have been done to classify a given comment or document as toxic or non-toxic. However, none of those proposed models work at the token level. In this paper, we propose a self-attention-based bidirectional gated recurrent unit(BiGRU) with a multi-embedding representation of the tokens. Our proposed model enriches the representation by a combination of GPT-2, GloVe, and RoBERTa embeddings, which led to promising results. Experimental results show that our proposed approach is very effective in detecting span tokens.