Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR

TL;DR

Character-R1 is a novel framework that improves role-aware reasoning in role-playing agents by providing structured, verifiable reward signals, leading to more consistent and effective behavior in complex scenarios.

Contribution

The paper introduces Character-R1, a new reward framework with three core designs that enhance internal cognition and role consistency in role-playing agents.

Findings

Significantly improves knowledge and memory performance.

Outperforms existing methods in role-aware reasoning.

Enhances internal cognitive consistency in complex situations.

Abstract

Current role-playing agents (RPAs) are typically constructed by imitating surface-level behaviors, but this approach lacks internal cognitive consistency, often causing out-of-character errors in complex situations. To address this, we propose Character-R1, a framework designed to provide comprehensive verifiable reward signals for effective role-aware reasoning, which are missing in recent studies. Specifically, our framework comprises three core designs: (1) Cognitive Focus Reward, which enforces explicit label-based analysis of 10 character elements (e.g., worldview) to structure internal cognition; (2) Reference-Guided Reward, which utilizes overlap-based metrics with reference responses as optimization anchors to enhance exploration and performance; and (3) Character-Conditioned Reward Normalization, which adjusts reward distributions based on character categories to ensure robust optimization across heterogeneous roles. Extensive experiments demonstrate that Character-R1 significantly outperforms existing methods in knowledge, memory and others.