HyCoRA: Hyper-Contrastive Role-Adaptive Learning for Role-Playing

TL;DR

HyCoRA introduces a novel hyper-contrastive role-adaptive learning framework that balances role-specific and shared traits, significantly enhancing multi-character role-playing in language models.

Contribution

The paper proposes a Hyper-Half Low-Rank Adaptation structure with a hyper-contrastive mechanism to better model distinct and shared role traits, improving role-playing abilities.

Findings

Outperforms existing methods on English and Chinese benchmarks.

Effectively captures role-specific personality signatures.

Validated by GPT-4 evaluations and visual analyses.

Abstract

Multi-character role-playing aims to equip models with the capability to simulate diverse roles. Existing methods either use one shared parameterized module across all roles or assign a separate parameterized module to each role. However, the role-shared module may ignore distinct traits of each role, weakening personality learning, while the role-specific module may overlook shared traits across multiple roles, hindering commonality modeling. In this paper, we propose a novel HyCoRA: Hyper-Contrastive Role-Adaptive learning framework, which efficiently improves multi-character role-playing ability by balancing the learning of distinct and shared traits. Specifically, we propose a Hyper-Half Low-Rank Adaptation structure, where one half is a role-specific module generated by a lightweight hyper-network, and the other half is a trainable role-shared module. The role-specific module is devised to represent distinct persona signatures, while the role-shared module serves to capture common traits. Moreover, to better reflect distinct personalities across different roles, we design a hyper-contrastive learning mechanism to help the hyper-network distinguish their unique characteristics. Extensive experimental results on both English and Chinese available benchmarks demonstrate the superiority of our framework. Further GPT-4 evaluations and visual analyses also verify the capability of HyCoRA to capture role characteristics.