Conversational Forecasting Across Large Human Groups Using A Swarm of Surrogate AI Agents

TL;DR

This study demonstrates that real-time conversational forecasting with Surrogate AI agents significantly improves group prediction accuracy on NBA game outcomes, outperforming Vegas odds and prediction markets.

Contribution

It introduces Hyperchat AI and Thinkscape platform for large human-AI collaborative forecasting, showing substantial accuracy improvements in sports predictions.

Findings

Teams achieved 62% accuracy on NBA forecasts using Hyperchat AI.

Forecast accuracy was positively correlated with conversation rate.

The approach outperformed Vegas odds and prediction markets in accuracy.

Abstract

Hyperchat AI is a communication and collaboration architecture that employs intervening AI agents to enable real-time conversational deliberations among networked human teams of unlimited size. Prior work has shown that teams as large as 250 people can hold productive real-time conversations by text, voice, or video using Hyperchat AI to discuss complex problems, brainstorm solutions, surface risks, assess alternatives, prioritize options, and converge on optimized results. Building on this prior work, this new study tasked groups of 25 to 30 basketball fans with conversationally forecasting NBA games (against the spread) over a 12-week period. Results show that when discussing and debating NBA games (for five minutes each) using a Hyperchat AI enabled platform called Thinkscape, human teams were 62% accurate across a set of 50 forecasted NBA games. This is an impressive result versus the Vegas odds of 50% (p=0.059). Furthermore, had the participants wagered on the games, they would have produced an 18.4% ROI over the 12-week period. In addition, this study found that the group's conversation rate during each forecast was positively correlated with their prediction accuracy. In fact, when excluding the 12 forecasts in the bottom 25th percentile by average conversation rate, the remaining 38 forecasts recorded a 68% accuracy, significantly better than the 50% Vegas odds (p=0.017). This result also outperformed the well-known prediction market Polymarket (p=0.062) across the same set of NBA games. These outcomes suggest that real-time conversational deliberations, when facilitated by Surrogate AI agents, can significantly amplify groupwise collective intelligence during human forecasting tasks.