AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactions

TL;DR

AgenticPay introduces a comprehensive multi-agent negotiation benchmark using natural language, enabling evaluation of LLMs in complex economic interactions with diverse tasks and realistic market scenarios.

Contribution

This work presents a novel benchmark and simulation framework for multi-agent negotiation with natural language, covering diverse market tasks and providing structured evaluation metrics.

Findings

State-of-the-art LLMs show significant gaps in negotiation performance.

Long-horizon strategic reasoning remains a key challenge.

AgenticPay enables studying language-based market interactions.

Abstract

Large language model (LLM)-based agents are increasingly expected to negotiate, coordinate, and transact autonomously, yet existing benchmarks lack principled settings for evaluating language-mediated economic interaction among multiple agents. We introduce AgenticPay, a benchmark and simulation framework for multi-agent buyer-seller negotiation driven by natural language. AgenticPay models markets in which buyers and sellers possess private constraints and product-dependent valuations, and must reach agreements through multi-round linguistic negotiation rather than numeric bidding alone. The framework supports a diverse suite of over 110 tasks ranging from bilateral bargaining to many-to-many markets, with structured action extraction and metrics for feasibility, efficiency, and welfare. Benchmarking state-of-the-art proprietary and open-weight LLMs reveals substantial gaps in negotiation performance and highlights challenges in long-horizon strategic reasoning, establishing AgenticPay as a foundation for studying agentic commerce and language-based market interaction. Code and dataset are available at the link: https://github.com/SafeRL-Lab/AgenticPay.