Socio-Culturally Aware Evaluation Framework for LLM-Based Content Moderation

TL;DR

This paper introduces a socio-culturally aware evaluation framework for LLM-based content moderation, emphasizing the importance of diverse, persona-driven datasets to improve assessment reliability and challenge smaller models.

Contribution

It presents a novel scalable method for creating diverse, persona-based datasets and demonstrates their effectiveness in revealing limitations of LLMs in content moderation.

Findings

Persona-based datasets offer broader perspectives.

Such datasets pose greater challenges for LLMs.

Smaller LLMs struggle more with diverse content.

Abstract

With the growth of social media and large language models, content moderation has become crucial. Many existing datasets lack adequate representation of different groups, resulting in unreliable assessments. To tackle this, we propose a socio-culturally aware evaluation framework for LLM-driven content moderation and introduce a scalable method for creating diverse datasets using persona-based generation. Our analysis reveals that these datasets provide broader perspectives and pose greater challenges for LLMs than diversity-focused generation methods without personas. This challenge is especially pronounced in smaller LLMs, emphasizing the difficulties they encounter in moderating such diverse content.