White Paper: Human-AI Collaboration in Conflict Analysis: Text Classifier Development with Peacebuilders

TL;DR

This paper presents a participatory approach to developing AI text classifiers for monitoring online polarization and hate speech, emphasizing collaboration with peacebuilders to enhance model relevance and cultural sensitivity.

Contribution

It demonstrates how collaborative annotation and validation improve AI model robustness, contextual accuracy, and practitioner ownership in sensitive humanitarian applications.

Findings

Models showed improved contextual alignment and reduced cultural misclassification.

Participatory development increased practitioner ownership of AI tools.

Open-source classifiers are available on HuggingFace.

Abstract

This paper documents a collaborative research process involving peacebuilders and data scientists in Kenya and Sudan to develop AI-based text classifiers for monitoring online polarization and hatespeech. The method describes a participatory annotation process in which practitioners and domain experts contributed to problem definition, annotation design, iterative validation, and model evaluation. Fine-tuned BERT-based classifiers were trained on collaboratively annotated datasets and evaluated against held-out test sets. In each case, the models produced enhanced contextual alignment, reduced misclassification driven by cultural nuance, and increased practitioner ownership of AI tools. The resulting models (Kenya-polarization and Sudan-hate speech) are open-source and accessible via HuggingFace. The study contributes empirical evidence that participatory AI development can simultaneously improve technical robustness, contextual validity, and normative alignment in sensitive humanitarian domains.