GRAID: Synthetic Data Generation with Geometric Constraints and Multi-Agentic Reflection for Harmful Content Detection

TL;DR

GRAID is a novel data augmentation pipeline that uses geometric constraints and multi-agent reflection with LLMs to improve harmful content detection by addressing data scarcity and exploring edge cases.

Contribution

GRAID introduces a two-stage augmentation process combining geometric control and multi-agent reflection to enhance harmful text classification datasets.

Findings

Significant performance improvements on benchmark datasets.

Enhanced stylistic diversity and edge case coverage.

Effective addressing of data scarcity in harmful content detection.

Abstract

We address the problem of data scarcity in harmful text classification for guardrailing applications and introduce GRAID (Geometric and Reflective AI-Driven Data Augmentation), a novel pipeline that leverages Large Language Models (LLMs) for dataset augmentation. GRAID consists of two stages: (i) generation of geometrically controlled examples using a constrained LLM, and (ii) augmentation through a multi-agentic reflective process that promotes stylistic diversity and uncovers edge cases. This combination enables both reliable coverage of the input space and nuanced exploration of harmful content. Using two benchmark data sets, we demonstrate that augmenting a harmful text classification dataset with GRAID leads to significant improvements in downstream guardrail model performance.