Persona Switch: Mixing Distinct Perspectives in Decoding Time

TL;DR

Persona Switch is a decoding technique that dynamically combines zero-shot and role-play prompting strategies, leveraging output confidence to improve language model performance across tasks.

Contribution

It introduces a novel step-by-step decoding method that adaptively chooses between prompting strategies based on confidence, outperforming existing baselines.

Findings

Achieves up to 5.13% accuracy improvement over baselines.

Output confidence effectively indicates the more reliable output.

Demonstrates consistent performance gains across different LLMs.

Abstract

Role-play prompting is known to steer the behavior of language models by injecting a persona into the prompt, improving their zero-shot reasoning capabilities. However, such improvements are inconsistent across different tasks or instances. This inconsistency suggests that zero-shot and role-play prompting may offer complementary strengths rather than one being universally superior. Building on this insight, we propose Persona Switch, a novel decoding method that dynamically combines the benefits of both prompting strategies. Our method proceeds step-by-step, selecting the better output between zero-shot and role-play prompting at each step by comparing their output confidence, as measured by the logit gap. Experiments with widely-used LLMs demonstrate that Persona Switch consistently outperforms competitive baselines, achieving up to 5.13% accuracy improvement. Furthermore, we show that output confidence serves as an informative measure for selecting the more reliable output.