LLM Agents for Bargaining with Utility-based Feedback

TL;DR

This paper introduces BargainArena, a new benchmark and a structured feedback mechanism to improve large language models' bargaining strategies by incorporating utility-based evaluation and opponent-aware reasoning, addressing real-world negotiation complexities.

Contribution

The paper presents a novel benchmark dataset, utility-based evaluation metrics, and a feedback mechanism to enhance LLMs' bargaining capabilities with strategic depth and opponent awareness.

Findings

Structured feedback improves LLM negotiation performance.

LLMs often misalign with human preferences in bargaining.

Feedback mechanism enhances opponent-aware reasoning.

Abstract

Bargaining, a critical aspect of real-world interactions, presents challenges for large language models (LLMs) due to limitations in strategic depth and adaptation to complex human factors. Existing benchmarks often fail to capture this real-world complexity. To address this and enhance LLM capabilities in realistic bargaining, we introduce a comprehensive framework centered on utility-based feedback. Our contributions are threefold: (1) BargainArena, a novel benchmark dataset with six intricate scenarios (e.g., deceptive practices, monopolies) to facilitate diverse strategy modeling; (2) human-aligned, economically-grounded evaluation metrics inspired by utility theory, incorporating agent utility and negotiation power, which implicitly reflect and promote opponent-aware reasoning (OAR); and (3) a structured feedback mechanism enabling LLMs to iteratively refine their bargaining strategies. This mechanism can positively collaborate with in-context learning (ICL) prompts, including those explicitly designed to foster OAR. Experimental results show that LLMs often exhibit negotiation strategies misaligned with human preferences, and that our structured feedback mechanism significantly improves their performance, yielding deeper strategic and opponent-aware reasoning.