Lattice: Generative Guardrails for Conversational Agents

TL;DR

Lattice is a framework that creates and continuously improves guardrails for conversational AI, enhancing safety by adapting to new threats through iterative simulation, risk assessment, and optimization.

Contribution

Lattice introduces a self-constructing, adaptive guardrail framework for conversational agents, surpassing static rule-based approaches with continuous improvement capabilities.

Findings

Achieves 91% F1 on ProsocialDialog dataset, outperforming baselines.

Demonstrates 7pp F1 improvement through closed-loop optimization.

Outperforms existing guardrails by significant margins.

Abstract

Conversational AI systems require guardrails to prevent harmful outputs, yet existing approaches use static rules that cannot adapt to new threats or deployment contexts. We introduce Lattice, a framework for self-constructing and continuously improving guardrails. Lattice operates in two stages: construction builds initial guardrails from labeled examples through iterative simulation and optimization; continuous improvement autonomously adapts deployed guardrails through risk assessment, adversarial testing, and consolidation. Evaluated on the ProsocialDialog dataset, Lattice achieves 91% F1 on held-out data, outperforming keyword baselines by 43pp, LlamaGuard by 25pp, and NeMo by 4pp. The continuous improvement stage achieves 7pp F1 improvement on cross-domain data through closed-loop optimization. Our framework shows that effective guardrails can be self-constructed through iterative optimization.