Enhancing Language Model Rationality with Bi-Directional Deliberation Reasoning

TL;DR

This paper presents BIDDER, a bi-directional reasoning method for language models that improves decision rationality by integrating past and future contexts, tested successfully in poker and negotiation scenarios.

Contribution

Introduces BIDDER, a novel bi-directional reasoning framework that enhances language model decision-making by incorporating future outcome prediction and historical context integration.

Findings

BIDDER significantly improves decision quality in poker.

BIDDER enhances negotiation decision-making.

Bi-directional reasoning outperforms uni-directional approaches.

Abstract

This paper introduces BI-Directional DEliberation Reasoning (BIDDER), a novel reasoning approach to enhance the decision rationality of language models. Traditional reasoning methods typically rely on historical information and employ uni-directional (left-to-right) reasoning strategy. This lack of bi-directional deliberation reasoning results in limited awareness of potential future outcomes and insufficient integration of historical context, leading to suboptimal decisions. BIDDER addresses this gap by incorporating principles of rational decision-making, specifically managing uncertainty and predicting expected utility. Our approach involves three key processes: Inferring hidden states to represent uncertain information in the decision-making process from historical data; Using these hidden states to predict future potential states and potential outcomes; Integrating historical information (past contexts) and long-term outcomes (future contexts) to inform reasoning. By leveraging bi-directional reasoning, BIDDER ensures thorough exploration of both past and future contexts, leading to more informed and rational decisions. We tested BIDDER's effectiveness in two well-defined scenarios: Poker (Limit Texas Hold'em) and Negotiation. Our experiments demonstrate that BIDDER significantly improves the decision-making capabilities of LLMs and LLM agents.