Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas

TL;DR

This paper explores how role-play techniques in large language models improve reasoning but also introduce ethical risks like bias and harmful outputs, highlighting the need for careful deployment.

Contribution

It reveals the paradoxical effects of role-play in LLMs, showing both reasoning benefits and increased ethical risks, especially with autotuning.

Findings

Role-play enhances reasoning capabilities in LLMs.

Autotuning can lead to harmful and biased outputs.

Role-play amplifies risks of stereotypes and harmful content.

Abstract

Role-play in large language models (LLMs) enhances their ability to generate contextually relevant and high-quality responses by simulating diverse cognitive perspectives. However, our study identifies significant risks associated with this technique. First, we demonstrate that autotuning, a method used to auto-select models' roles based on the question, can lead to the generation of harmful outputs, even when the model is tasked with adopting neutral roles. Second, we investigate how different roles affect the likelihood of generating biased or harmful content. Through testing on benchmarks containing stereotypical and harmful questions, we find that role-play consistently amplifies the risk of biased outputs. Our results underscore the need for careful consideration of both role simulation and tuning processes when deploying LLMs in sensitive or high-stakes contexts.