Assessing and Refining ChatGPT's Performance in Identifying Targeting and Inappropriate Language: A Comparative Study

TL;DR

This study evaluates ChatGPT's ability to detect targeting and inappropriate language in online comments, comparing its performance to human annotations, and highlights its strengths and areas needing improvement for content moderation.

Contribution

It provides a comparative analysis of ChatGPT's effectiveness in moderation tasks and demonstrates the impact of iterative refinements on its accuracy.

Findings

ChatGPT performs well in detecting inappropriate content.

Iterative refinements improve detection accuracy.

Higher false positives in targeting language detection.

Abstract

This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate language in online comments. With the increasing challenge of moderating vast volumes of user-generated content on social network sites, the role of AI in content moderation has gained prominence. We compared ChatGPT's performance against crowd-sourced annotations and expert evaluations to assess its accuracy, scope of detection, and consistency. Our findings highlight that ChatGPT performs well in detecting inappropriate content, showing notable improvements in accuracy through iterative refinements, particularly in Version 6. However, its performance in targeting language detection showed variability, with higher false positive rates compared to expert judgments. This study contributes to the field by demonstrating the potential of AI models like ChatGPT to enhance automated content moderation systems while also identifying areas for further improvement. The results underscore the importance of continuous model refinement and contextual understanding to better support automated moderation and mitigate harmful online behavior.