RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation

TL;DR

RoTRAG is a retrieval-augmented framework that enhances harm detection in multi-turn dialogues by explicitly incorporating external moral norms, improving accuracy and efficiency over existing methods.

Contribution

It introduces a novel retrieval-based approach using Rules of Thumb for normative reasoning in harm detection, with a lightweight classifier for efficiency.

Findings

RoTRAG improves harm classification and severity estimation accuracy.

It achieves around 40% relative gain in F1 score on benchmark datasets.

It reduces redundant computation by 8.4% without performance loss.

Abstract

Detecting harmful content in multi turn dialogue requires reasoning over the full conversational context rather than isolated utterances. However, most existing methods rely mainly on models internal parametric knowledge, without explicit grounding in external normative principles. This often leads to inconsistent judgments in socially nuanced contexts, limited interpretability, and redundant reasoning across turns. To address this, we propose RoTRAG, a retrieval augmented framework that incorporates concise human written moral norms, called Rules of Thumb (RoTs), into LLM based harm assessment. For each turn, RoTRAG retrieves relevant RoTs from an external corpus and uses them as explicit normative evidence for turn level reasoning and final severity classification. To improve efficiency, we further introduce a lightweight binary routing classifier that decides whether a new turn requires retrieval grounded reasoning or can reuse existing context. Experiments on ProsocialDialog and Safety Reasoning Multi Turn Dialogue show that RoTRAG consistently improves both harm classification and severity estimation over competitive baselines, with an average relative gain of around 40% in F1 across benchmark datasets and an average relative reduction of 8.4% in distributional error, while reducing redundant computation without sacrificing performance.