Efficient Toxicity Detection in Gaming Chats: A Comparative Study of Embeddings, Fine-Tuned Transformers and LLMs

TL;DR

This study compares various NLP techniques, including embeddings, LLMs, and fine-tuned transformers, for toxicity detection in gaming chats, highlighting the most effective methods balancing accuracy and efficiency.

Contribution

It offers a comprehensive evaluation of multiple NLP approaches for toxicity detection and proposes a hybrid moderation system to optimize moderation workload.

Findings

Fine-tuned DistilBERT achieves best accuracy-cost balance.

Significant performance differences observed among methods.

Hybrid system reduces human moderator workload.

Abstract

This paper presents a comprehensive comparative analysis of Natural Language Processing (NLP) methods for automated toxicity detection in online gaming chats. Traditional machine learning models with embeddings, large language models (LLMs) with zero-shot and few-shot prompting, fine-tuned transformer models, and retrieval-augmented generation (RAG) approaches are evaluated. The evaluation framework assesses three critical dimensions: classification accuracy, processing speed, and computational costs. A hybrid moderation system architecture is proposed that optimizes human moderator workload through automated detection and incorporates continuous learning mechanisms. The experimental results demonstrate significant performance variations across methods, with fine-tuned DistilBERT achieving optimal accuracy-cost trade-offs. The findings provide empirical evidence for deploying cost-effective, efficient content moderation systems in dynamic online gaming environments.