FBERT: A Neural Transformer for Identifying Offensive Content

TL;DR

fBERT is a retrained BERT model on a large offensive language dataset, achieving improved performance in identifying offensive content in social media, and is made publicly available.

Contribution

The paper introduces fBERT, a BERT model retrained on the largest English offensive language dataset, enhancing offensive content detection.

Findings

fBERT outperforms previous models on multiple datasets

Optimal thresholds improve offensive content classification

fBERT is publicly accessible for research use

Abstract

Transformer-based models such as BERT, XLNET, and XLM-R have achieved state-of-the-art performance across various NLP tasks including the identification of offensive language and hate speech, an important problem in social media. In this paper, we present fBERT, a BERT model retrained on SOLID, the largest English offensive language identification corpus available with over $1.4$ million offensive instances. We evaluate fBERT's performance on identifying offensive content on multiple English datasets and we test several thresholds for selecting instances from SOLID. The fBERT model will be made freely available to the community.