LT@Helsinki at SemEval-2020 Task 12: Multilingual or language-specific BERT?

TL;DR

This paper describes how the LT@Helsinki team used BERT models fine-tuned on specific datasets to achieve state-of-the-art results in multilingual offensive language detection tasks at SemEval-2020.

Contribution

The paper demonstrates the effectiveness of BERT for multilingual offensive language identification and provides models fine-tuned for specific sub-tasks.

Findings

BERT achieved state-of-the-art results in offensive language detection.

Fine-tuning BERT on OLID and SOLID datasets is effective.

Multilingual BERT models perform well across different languages.

Abstract

This paper presents the different models submitted by the LT@Helsinki team for the SemEval 2020 Shared Task 12. Our team participated in sub-tasks A and C; titled offensive language identification and offense target identification, respectively. In both cases we used the so-called Bidirectional Encoder Representation from Transformer (BERT), a model pre-trained by Google and fine-tuned by us on the OLID and SOLID datasets. The results show that offensive tweet classification is one of several language-based tasks where BERT can achieve state-of-the-art results.