Exchange of Perspective Prompting Enhances Reasoning in Large Language Models
Lin Sun, Can Zhang
TL;DR
This paper introduces Exchange-of-Perspective (EoP), a framework that exchanges problem perspectives to improve reasoning in large language models, showing significant performance gains across multiple NLP benchmarks.
Contribution
The paper proposes a novel EoP framework that enhances LLM reasoning by exchanging perspectives, demonstrating substantial improvements on 8 benchmark datasets.
Findings
EoP improves GPT-3.5-Turbo performance by 3.6% on AQuA.
GPT-4 with EoP shows a 7.7% accuracy increase on Math benchmarks.
EoP enhances OlympiadBench Maths accuracy by 3.5%.
Abstract
Large language models (LLMs) have made significant advancements in addressing diverse natural language processing (NLP) tasks. However, their performance is often limited by inherent comprehension of problems. To address this limitation, we propose Exchange-of-Perspective (EoP), a novel framework designed to exchange perspectives across different definitions of problem, so that it can break the fixed mindset from any particular formulation of the question. We conducted extensive and comprehensive experiments on 8 benchmarks. The results show that EoP can significantly improve performance. For instance, compared to the non-commutative baseline PHP, with GPT-3.5-Turbo and EoP, we observe a 3.6% improvement on AQuA (60.6% to 64.2%), while GPT-4-powered EoP demonstrates a 7.7% overall accuracy enhancement on Math (53.9% to 61.6%) and a 3.5% improvement on OlympiadBench Maths (43.5% to…
Peer Reviews
Decision·Submitted to ICLR 2025
- The motivation of exploiting different definitions of the same question is insightful. This focus on the question on the input side, rather than the reasoning process on the generation side, opens up a new direction for improving LLM performance, which can be impactful. - This framework further exchanges reasoning processes across two questions, so that it can break the fixed mindset from any particular formulation of the question, leading to more robust and accurate answers. This idea is als
- This work is a bit incremental compared to PHP. The all use previous answers to refine the answer in multiple iterations, and the difference is that EoP uses prior answers from the other branch as hints. - Due to the iterative and two-branch nature of this method, this will significantly increase the cost. - The improvements of this method for arithmetic reasoning in Table 1 and Table 2 look marginal compared with the previous state-of-the-art methods. - The choice of datasets is narrow bec
The main strength of the paper is its introduction of EoP as a novel framework to improve reasoning in LLMs. This paper focuses on reformulating questions externally, allowing cross-checking of answers through feedback loops from alternate perspectives, rather than relying on internal logic as seen in methods such as CoT. This design shows that creating augmented branches and proceeding with EoP can improve reasoning accuracy and surpass existing baselines in reasoning datasets. By integrating f
The EoP framework relies on an iterative feedback loop until a termination condition is met, which results in a significant increase in computational cost compared to CoT prompting. The paper does not appear to discuss computational cost in detail, given that only GPT models are used. Solely using GPT may not be sufficient to showcase how EoP outperforms other existing baselines. For instance, it is unclear how open-source models like llama, Qwen2, among others, would perform under the EoP frame
The strengths of this paper are listed as follows: 1. The paper is well organized in structure, which helps me better understand the research ideas of the authors. 2. The experiments are detailed, covering multiple tasks and multiple different datasets. EoP is compared with multiple baselines.
The weaknesses of this paper are listed as follows: 1. **Redundant Expressions.** I don't like the expressions in Section 2.1. I think the explanations of the method EoP is too redundant and complex. These cumbersome symbols and formulas do not help me better understand how EoP is implemented, but instead add to the burden of reading. I think it is better to simplify this section. 2. **Limited Novelty.** Actually, it is not surprising that the accuracy of the model's answers can be improved by r
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Computational and Text Analysis Methods · Advanced Graph Neural Networks