Information Aggregation with AI Agents

TL;DR

This paper investigates how AI agents aggregate private information in prediction markets, revealing limitations with complex information and showing robustness to various market modifications.

Contribution

It demonstrates that AI agents' information aggregation is effective in simple settings but diminishes with complexity, and identifies factors influencing performance.

Findings

AI agents effectively aggregate information in simple structures

Complex information reduces aggregation effectiveness

Market modifications do not significantly affect aggregation

Abstract

Can Large Language Models (AI agents) aggregate dispersed private information through trading and reason about the knowledge of others by observing price movements? We conduct a controlled experiment where AI agents trade in a prediction market after receiving private signals, measuring information aggregation by the log error of the last price. We find that although the median market is effective at aggregating information in the easy information structures, increasing the complexity has a significant and negative impact, suggesting that AI agents may suffer from similar limitations as humans when reasoning about others. Consistent with our theoretical predictions, information aggregation remains unaffected by allowing cheap talk communication, changing the duration of the market or initial price, and strategic prompting, thus demonstrating that prediction markets are robust. We establish that "smarter" AI agents perform better at aggregation and they are more profitable. Surprisingly, giving them feedback about past performance has no impact on aggregation.