Natural Language Mechanisms via Self-Resolution with Foundation Models

TL;DR

This paper introduces a new class of mechanisms that use natural language reports and large language models to improve outcome selection and payoff assignment, surpassing traditional prediction markets in certain scenarios.

Contribution

It proposes a novel framework combining natural language elicitation with LLMs, establishing conditions for incentive compatibility and efficiency in information aggregation.

Findings

LLM-based mechanisms can effectively aggregate information where prediction markets fail.

Conditions for incentive compatibility include the LLM being a good enough world model.

Demonstrates scenarios where natural language mechanisms outperform traditional methods.

Abstract

Practical mechanisms often limit agent reports to constrained formats like trades or orderings, potentially limiting the information agents can express. We propose a novel class of mechanisms that elicit agent reports in natural language and leverage the world-modeling capabilities of large language models (LLMs) to select outcomes and assign payoffs. We identify sufficient conditions for these mechanisms to be incentive-compatible and efficient as the LLM being a good enough world model and a strong inter-agent information over-determination condition. We show situations where these LM-based mechanisms can successfully aggregate information in signal structures on which prediction markets fail.