Better Zero-Shot Reasoning with Role-Play Prompting
Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin, Zhou, Enzhi Wang, Xiaohang Dong

TL;DR
This paper introduces role-play prompting to enhance zero-shot reasoning in large language models, demonstrating significant performance improvements across multiple benchmarks and outperforming existing techniques like Zero-Shot-CoT.
Contribution
The study presents a novel role-play prompting method that effectively boosts zero-shot reasoning abilities in LLMs, validated through extensive empirical evaluation.
Findings
Role-play prompting outperforms standard zero-shot across most datasets.
Accuracy on AQuA increases from 53.5% to 63.8%.
Accuracy on Last Letter increases from 23.8% to 84.2%.
Abstract
Modern large language models (LLMs) exhibit a remarkable capacity for role-playing, enabling them to embody not only human characters but also non-human entities. This versatility allows them to simulate complex human-like interactions and behaviors within various contexts, as well as to emulate specific objects or systems. While these capabilities have enhanced user engagement and introduced novel modes of interaction, the influence of role-playing on LLMs' reasoning abilities remains underexplored. In this study, we introduce a strategically designed role-play prompting methodology and assess its performance under the zero-shot setting across twelve diverse reasoning benchmarks. Our empirical results illustrate that role-play prompting consistently surpasses the standard zero-shot approach across most datasets. Notably, in experiments conducted using ChatGPT, accuracy on AQuA rises…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsLogic, Reasoning, and Knowledge · Topic Modeling · Multi-Agent Systems and Negotiation
