xList-Hate: A Checklist-Based Framework for Interpretable and Generalizable Hate Speech Detection

TL;DR

xList-Hate introduces a diagnostic, checklist-based framework using large language models and decision trees to improve interpretability and robustness in hate speech detection across diverse datasets.

Contribution

The paper presents a novel diagnostic framework that decomposes hate speech detection into explicit questions answered by LLMs, enhancing interpretability and robustness over traditional classification methods.

Findings

Improves cross-dataset robustness compared to supervised models.

Provides transparent, interpretable decision paths.

Reduces sensitivity to annotation noise and ambiguity.

Abstract

Hate speech detection is commonly framed as a direct binary classification problem despite being a composite concept defined through multiple interacting factors that vary across legal frameworks, platform policies, and annotation guidelines. As a result, supervised models often overfit dataset-specific definitions and exhibit limited robustness under domain shift and annotation noise. We introduce xList-Hate, a diagnostic framework that decomposes hate speech detection into a checklist of explicit, concept-level questions grounded in widely shared normative criteria. Each question is independently answered by a large language model (LLM), producing a binary diagnostic representation that captures hateful content features without directly predicting the final label. These diagnostic signals are then aggregated by a lightweight, fully interpretable decision tree, yielding transparent and auditable predictions. We evaluate it across multiple hate speech benchmarks and model families, comparing it against zero-shot LLM classification and in-domain supervised fine-tuning. While supervised methods typically maximize in-domain performance, we consistently improves cross-dataset robustness and relative performance under domain shift. In addition, qualitative analysis of disagreement cases provides evidence that the framework can be less sensitive to certain forms of annotation inconsistency and contextual ambiguity. Crucially, the approach enables fine-grained interpretability through explicit decision paths and factor-level analysis. Our results suggest that reframing hate speech detection as a diagnostic reasoning task, rather than a monolithic classification problem, provides a robust, explainable, and extensible alternative for content moderation.