KEIS@JUST at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT

TL;DR

This paper describes KEIS@JUST's participation in SemEval-2020 Task 12, employing weighted ensemble models and fine-tuned BERT for multilingual offensive tweet detection across various languages.

Contribution

The study introduces a novel ensemble approach for Arabic and English and applies transfer learning with BERT for other languages, enhancing multilingual offensive language identification.

Findings

Weighted ensemble improved performance for Arabic and English.

Transfer learning with BERT enhanced detection in other languages.

Data augmentation benefited Arabic language classification.

Abstract

This research presents our team KEIS@JUST participation at SemEval-2020 Task 12 which represents shared task on multilingual offensive language. We participated in all the provided languages for all subtasks except sub-task-A for the English language. Two main approaches have been developed the first is performed to tackle both languages Arabic and English, a weighted ensemble consists of Bi-GRU and CNN followed by Gaussian noise and global pooling layer multiplied by weights to improve the overall performance. The second is performed for other languages, a transfer learning from BERT beside the recurrent neural networks such as Bi-LSTM and Bi-GRU followed by a global average pooling layer. Word embedding and contextual embedding have been used as features, moreover, data augmentation has been used only for the Arabic language.