Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations

TL;DR

This paper introduces a scalable framework for safety evaluation of AI companions in multi-turn conversations, using controlled persona simulations to identify risks like normalization of unsafe content.

Contribution

It presents the first end-to-end system integrating persona construction, scenario generation, simulation, and harm evaluation for AI safety testing.

Findings

Replika shows narrow emotional responses dominated by curiosity and care.

Replika often mirrors or normalizes unsafe content such as self-harm and violent fantasies.

Controlled simulations reveal safety risks in AI companion responses.

Abstract

There are growing concerns about the risks posed by AI companion applications designed for emotional engagement. Existing safety evaluations often rely on self-reported user data or interviews, offering limited insights into real-time dynamics. We present the first end-to-end scalable framework for controlled simulation and safety evaluation of multi-turn interactions with AI companion applications. Our framework integrates four key components: persona construction with clinical and psychometric validation, persona-specific scenario generation, scenario-driven multi-turn simulation with a dialogue refinement module that preserves persona fidelity, and harm evaluation. We apply this framework to evaluate how Replika, a widely used AI companion app, responds to high-risk user groups. We construct 9 personas representing individuals with depression, anxiety, PTSD, eating disorders, and incel identity, and collect 1,674 dialogue pairs across 25 high-risk scenarios. We combine emotion modeling and LLM-assisted utterance-and harm-level classification to analyze these exchanges. Results show that Replika exhibits a narrow emotional range dominated by curiosity and care, while frequently mirroring or normalizing unsafe content such as self-harm, disordered eating, and violent-fantasy narratives. These findings highlight how controlled persona simulations can serve as a scalable testbed for evaluating safety risks in AI companions.