Toxic Comments Hunter : Score Severity of Toxic Comments

TL;DR

This paper presents a system for scoring the severity of toxic comments by collecting datasets, performing data cleaning and feature extraction, and training models using TFIDF and fine-tuned BERT, enabling real-time toxicity scoring.

Contribution

It introduces a comprehensive approach combining traditional and deep learning models for toxic comment severity assessment with real-time implementation.

Findings

Effective data cleaning and feature extraction methods for toxic comments

BERT-based models outperform traditional TFIDF models in severity scoring

Real-time toxicity scoring system successfully deployed

Abstract

The detection and identification of toxic comments are conducive to creating a civilized and harmonious Internet environment. In this experiment, we collected various data sets related to toxic comments. Because of the characteristics of comment data, we perform data cleaning and feature extraction operations on it from different angles to obtain different toxic comment training sets. In terms of model construction, we used the training set to train the models based on TFIDF and finetuned the Bert model separately. Finally, we encapsulated the code into software to score toxic comments in real-time.