FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Models

TL;DR

FanarGuard is a bilingual moderation filter for Arabic and English that incorporates cultural awareness, constructed from a large dataset, and evaluated with a new Arabic cultural benchmark, improving alignment with human judgments.

Contribution

This work introduces FanarGuard, the first culturally-aware moderation filter for Arabic language models, and develops a novel benchmark for evaluating cultural alignment in moderation.

Findings

FanarGuard outperforms existing filters in cultural alignment with human judgments.

It matches state-of-the-art safety performance on safety benchmarks.

The benchmark reveals the importance of cultural context in moderation.

Abstract

Content moderation filters are a critical safeguard against alignment failures in language models. Yet most existing filters focus narrowly on general safety and overlook cultural context. In this work, we introduce FanarGuard, a bilingual moderation filter that evaluates both safety and cultural alignment in Arabic and English. We construct a dataset of over 468K prompt and response pairs, drawn from synthetic and public datasets, scored by a panel of LLM judges on harmlessness and cultural awareness, and use it to train two filter variants. To rigorously evaluate cultural alignment, we further develop the first benchmark targeting Arabic cultural contexts, comprising over 1k norm-sensitive prompts with LLM-generated responses annotated by human raters. Results show that FanarGuard achieves stronger agreement with human annotations than inter-annotator reliability, while matching the performance of state-of-the-art filters on safety benchmarks. These findings highlight the importance of integrating cultural awareness into moderation and establish FanarGuard as a practical step toward more context-sensitive safeguards.