Constructive and Toxic Speech Detection for Open-domain Social Media Comments in Vietnamese

TL;DR

This paper introduces a new Vietnamese dataset for detecting constructive and toxic comments on social media, and proposes a system using PhoBERT that achieves promising classification performance.

Contribution

It creates the first large-scale Vietnamese dataset for speech detection and develops a transfer learning-based system for automatic comment classification.

Findings

Achieved F1-score of 78.59% for constructive comment detection.

Achieved F1-score of 59.40% for toxic comment detection.

Evaluated baseline models to benchmark performance.

Abstract

The rise of social media has led to the increasing of comments on online forums. However, there still exists invalid comments which are not informative for users. Moreover, those comments are also quite toxic and harmful to people. In this paper, we create a dataset for constructive and toxic speech detection, named UIT-ViCTSD (Vietnamese Constructive and Toxic Speech Detection dataset) with 10,000 human-annotated comments. For these tasks, we propose a system for constructive and toxic speech detection with the state-of-the-art transfer learning model in Vietnamese NLP as PhoBERT. With this system, we obtain F1-scores of 78.59% and 59.40% for classifying constructive and toxic comments, respectively. Besides, we implement various baseline models as traditional Machine Learning and Deep Neural Network-Based models to evaluate the dataset. With the results, we can solve several tasks on the online discussions and develop the framework for identifying constructiveness and toxicity of Vietnamese social media comments automatically.