KoMultiText: Large-Scale Korean Text Dataset for Classifying Biased Speech in Real-World Online Services

TL;DR

KoMultiText is a large-scale Korean dataset with multi-label annotations for bias, profanity, and preferences, enabling advanced multi-task classification to detect harmful online speech and improve community health.

Contribution

The paper introduces KoMultiText, a comprehensive Korean dataset with multi-task annotations, and demonstrates state-of-the-art BERT-based models surpassing human accuracy in bias detection.

Findings

BERT-based models outperform human-level accuracy in classification tasks.

KoMultiText enables multi-task learning for bias, profanity, and preference detection.

The dataset is publicly available for research and practical applications.

Abstract

With the growth of online services, the need for advanced text classification algorithms, such as sentiment analysis and biased text detection, has become increasingly evident. The anonymous nature of online services often leads to the presence of biased and harmful language, posing challenges to maintaining the health of online communities. This phenomenon is especially relevant in South Korea, where large-scale hate speech detection algorithms have not yet been broadly explored. In this paper, we introduce "KoMultiText", a new comprehensive, large-scale dataset collected from a well-known South Korean SNS platform. Our proposed dataset provides annotations including (1) Preferences, (2) Profanities, and (3) Nine types of Bias for the text samples, enabling multi-task learning for simultaneous classification of user-generated texts. Leveraging state-of-the-art BERT-based language models, our approach surpasses human-level accuracy across diverse classification tasks, as measured by various metrics. Beyond academic contributions, our work can provide practical solutions for real-world hate speech and bias mitigation, contributing directly to the improvement of online community health. Our work provides a robust foundation for future research aiming to improve the quality of online discourse and foster societal well-being. All source codes and datasets are publicly accessible at https://github.com/Dasol-Choi/KoMultiText.